CODEWITHSUNDEEP
rcsvlapply
J.VDW
J.VDW

Reputation: 101

R extract a single data from mutliple csv files

I have multiple csv files (more than 100). Each file represents a time period. In each file, there are 29 lines that need to be skiped (text lines). At line 30, I have a matrix of temperature as a function of latitude and longitude coordinates. For example: at latitude 68.80 and longitude 48.40268, the temperature is 5.94.

So, I would like to extract the temperature at a specific combination of latitude and longitude coordinates for every time period I have (for every file).

I can write the code for a single file, but I'm affraid I don't know how to do it in a loop or how to make it faster.

Any help is appreciated, thank you. Sorry if this is similar to other questions, I read what I could find on that topic, but it did not seem to fit for my problem.

The code for one file:

filenames <- list.files(path="E:/Documents...")
fileone <- read.csv(filenames[1], skip=29, header=T, sep=";")
names(fileone) <- c("Lat", "68.88", "68.86", "68.85", "68.83", "68.82", "68.80", "68.79", "68.77", "68.76", "68.74", "68.73", "68.71")
Tempone <- fileone[which(fileone$Lat==48.40268), "68.80"]

Upvotes: 1

Views: 3391

Answers (1)

Ricardo Saporta
Ricardo Saporta

Reputation: 55340

Assuming data size relative to your system are small enough (relative to your system) to fit into memory all together, you can accomplish this in one shot using lists

## Grab the filienames, just like you're doing
filenames <- list.files(path="E:/Documents...")

## Assuming all columns have the same column names
c.nms <- c("Lat", "68.88", "68.86", "68.85", "68.83", "68.82", "68.80", "68.79", "68.77", "68.76", "68.74", "68.73", "68.71")


## Import them all in one shot, as a list of data.frames
AllData <- lapply(filenames, read.table, 
      skip=29, header=TRUE, sep=";", row.names=NULL, col.names=c.nms)

## Then to get all of your rows
PulledRows <- 
lapply(AllData, function(DF) 
    DF[fileone$Lat==48.40268, "68.80"]
  )

If you are pulling different lat/longs per file, you can use mapply with a vector/list of lat/longs

Upvotes: 3

Related Questions