CODEWITHSUNDEEP
rexcel
Mai
Mai

Reputation: 1

Splitting a csv file into multiple txt. files

I have a large csv dataset that I want to split into multiple txt files. I want the name of each file to come from the ID column and the content of each file to come from the Text column. My data looks something like this.

ID      Text
1       I like dogs
2       My name is
3       It is sunny

Would anyone be able to help advise? I don't mind using excel or R. Thank you!

Upvotes: 0

Views: 812

Answers (2)

Prayag Gordy
Prayag Gordy

Reputation: 777

If you have a lot of rows, this is a good task for parallel computing. (Here's the general premise: R spends a lot of time formatting the file. Writing to the disk can't be done in parallel, but formatting the file can.) So let's do it in parallel!

The furrr package is one of my favorites: In short, it adds parallel processing capabilities to the purrr package, whose map functions are quite useful. In this case, we want to use the future_pmap function, which lets us apply a function to each row of a dataframe. This should be all the code you need:

library(furrr)
plan(multiprocess)
future_pmap(df, function(id, value) {write(value, paste0(id, ".txt"))})

I tested the parallel and normal versions of this function on a dataframe with 31,496 rows, and the parallel version took only 60 percent as long. This method is also about 20 percent faster than Ronak Shah's writeLines method.

Upvotes: 2

Ronak Shah
Ronak Shah

Reputation: 388982

In R, You can split the data by ID and use writeLines to write it in text files.

If your dataframe is called df, try : 

temp <- split(df$Text, df$ID)
Map(function(x, y) writeLines(x, paste0(y, '.txt')), temp, names(temp))

Upvotes: 3

Related Questions