CODEWITHSUNDEEP
rdataframerbind
Umi
Umi

Reputation: 125

R: import identically formatted data, make a column corresponding to the original filename, vertically concatenate?

I am working with some ocean sensors that were deployed at different depths. Each sensor recorded several parameters (time, temperature, oxygen) at different depths, and each outputted an identically formatted file which I have renamed to 'top.csv', 'mid.csv', bot.csv' (for top, middle, bottom).

I currently have only three files, but will eventually have more so I want to set this up iteratively. Optimally I would have something set up such that:

  1. R will import all csv files from a specified directory
  2. It will add a column to each data frame called "depth" with the name of the original file.
  3. rbind them into a single data frame.

I am able to do steps 1 and 3 with the two lines below. The first line gets the file names from a specific directory that match the pattern, while the second line uses lapply nested in do.call to read all the files and vertically concatenate.

files = list.files('./data/', pattern="*.csv")

oxygenData= do.call(rbind, lapply(files, function(x) read.csv(paste('./data/',x)))

The justification to end up with a single data file is to plot them easier, as such: ggplot(data = oxygenData, aes(x = time, y = oxygen, group = depth, color = depth))+geom_line()

Also, would dealing with this kind of data be easier with data.table? Thank you!

Upvotes: 0

Views: 56

Answers (1)

lmo
lmo

Reputation: 38520

You can accomplish this by building your own function:

myFunc <- function(fileName) {
  # read in file
  temp <- read.csv(paste0("<filePath>/", fileName), as.is=TRUE)
  # assign file name
  temp$fileName <- fileName
  # return data.frame
  temp
}

Note that you could generalize myFunc by adding a second argument that takes the file path, allowing the directory to be set dynamically. Next, put this into lapply to get a list of data.frames:

myList <- lapply(fileNameVector, myFunc)

Finally, append the files using do.call and rbind.

res <- do.call(rbind, myList)

Upvotes: 1

Related Questions