CODEWITHSUNDEEP
r
Achak
Achak

Reputation: 1296

How to create single table by extracting certain cells from multiple CSV files

I am wondering if it is possible to create a new dataframe with certain cells from each file from the working directory. for example say If I have 2 data frame like this (please ignore the numbers as they are random):

enter image description here

Say in each dataset, row 4 is the sum of my value and Row 5 is number of missing values. If I represent number of missing values as "M" and Sum of coloumns as "N", what I am trying to acheive is the following table:

enter image description here

So each file 'N' and 'M' are in 1 single row.

I have many files in the directory so I have read them in a list, but not sure what would be the best way to perform such task on a list of files.

this is my sample code for the tables I have shown and how I read them in list:

 ##Create sample data

df = data.frame(Type = 'wind', v1=c(1,2,3,100,50), v2=c(4,5,6,200,60), v3=c(6,7,8,300,70))
df2 =data.frame(Type = 'test', v1=c(3,2,1,400,40), v2=c(2,3,4,500,30), v3=c(6,7,8,600,20))

# write to directory
write.csv(df, file = "sample1.csv", row.names = F)
write.csv(df2, file = "sample2.csv", row.names = F)

# read to list
mycsv = dir(pattern=".csv")
n <- length(mycsv) 
 
mylist <- vector("list", n) 
for(i in 1:n) mylist[[i]] <- read.csv(mycsv[i],header = TRUE)

I would be really greatful if you could give me some suggestion about if this possible and how I should approch?

Many thanks,
Ayan

Upvotes: 0

Views: 1253

Answers (2)

Josh O&#39;Brien
Josh O&#39;Brien

Reputation: 162321

This should work:

processFile <- function(File) {
    d <- read.csv(File, skip = 4, nrows = 2, header = FALSE, 
                  stringsAsFactors = FALSE)
    dd <- data.frame(d[1,1], t(unlist(d[-1])))
    names(dd) <- c("ID", "v1N", "V1M", "v2N", "V2M", "v3N", "V3M") 
    return(dd)
}

ll <- lapply(mycsv, processFile)
do.call(rbind, ll)
#     ID v1N V1M v2N V2M v3N V3M
# 1 wind 100  50 200  60 300  70
# 2 test 400  40 500  30 600  20

(The one slightly tricky/unusual bit comes in that third line of processFile(). Here's a code snippet that should help you see how it accomplishes what it does.)

(d <- data.frame(a="wind", b=1:2, c=3:4))
#      a b c
# 1 wind 1 3
# 2 wind 2 4
t(unlist(d[-1]))
#      b1 b2 c1 c2
# [1,]  1  2  3  4

Upvotes: 1

Tyler Rinker
Tyler Rinker

Reputation: 109844

CAVEAT: I'm not sure I fully understand what you want. I think you're reading in a list and want to select certain dataframes from that list with the same rows from that list. Then you want to create a data frame of those rows and go from long to wide format.

LIST <- lapply(2:3, function(i) {
        x <- mylist[[i]][4:5, ]
        x <- data.frame(x, row = factor(rownames(x)))
        return(x)
    }
)

DF <- do.call("rbind", LIST)  #lets you bind an unknown number of rows from a list
levels(DF$row) <- list(M =4, N = 5) #recodes rows 4 and 5 with M and N

wide <- reshape(DF, v.names=c("v1", "v2", "v3"), idvar=c("Type"),
    timevar="row", direction="wide") #reshape from long to wide

rownames(wide) <- 1:nrow(wide) #give proper row names
wide

This yields:

  Type v1.M v2.M v3.M v1.N v2.N v3.N
1 wind  100  200  300   50   60   70
2 test  400  500  600   40   30   20

Upvotes: 1

Related Questions