CODEWITHSUNDEEP
rfor-loopdata.tablesapply
89_Simple
89_Simple

Reputation: 3805

Apply a function to a list of csv files

I have 45 csv files in a folder called myFolder. Each csv file has 13 columns and 640 rows.

I want to read each csv and divide the columns 7:12 by 10 and save it in a new folder called 'my folder'. Here's my appraoch which is using the simple for loop.

library(data.table)
dir.create('newFolder')

allFiles <- list.files(file.path('myFolder'), pattern = '.csv')

for(a in seq_along(allFiles)){

    fileRef <- allFiles[a]
    temp <- fread(file.path('myFolder', fileRef)
    temp[, 7:12] <- temp[, 7:12]/10
    fwrite(temp, file.path('myFolder', paste0('new_',fileRef)))
 }

Is there a more simple solution in a line or two using datatable and apply function to achieve this?

Upvotes: 1

Views: 522

Answers (2)

G. Grothendieck
G. Grothendieck

Reputation: 270288

Your code is already pretty good but these improvements could be made:

  • define the input and output folders up front for modularity
  • use full.names = TRUE so that allFiles contains complete paths
  • use .csv$ as the pattern to anchor it to the end of the filename
  • iterate over the full names rather than an index
  • use basename in fwrite to extract out the base name from the path name

The code is then

library(data.table)

myFolder <- "myFolder"
newFolder <- "newFolder"

dir.create(newFolder)
allFiles <- list.files(myFolder, pattern = '.csv$', full.names = TRUE)

for(f in allFiles) {
    temp <- fread(f)
    temp[, 7:12] <- temp[, 7:12] / 10
    fwrite(temp, file.path(newFolder, paste0('new_', basename(f))))
}

Upvotes: 2

slava-kohut
slava-kohut

Reputation: 4233

You can use purrr::walk if you want to improve readability of your code and get rid of the loop:

allFiles <- list.files(file.path('myFolder'), pattern = '.csv')

purrr::walk(allFiles, function(x){
  temp <- fread(file.path('myFolder', x)
  temp[, 7:12] <- temp[, 7:12]/10
  fwrite(temp, file.path('myFolder', paste0('new_',fileRef)))
})

From the reference page of purrr::walk:

walk() returns the input .x (invisibly)

I don't think it helps speed-wise, though.

Upvotes: 1

Related Questions