CODEWITHSUNDEEP
rdirectorycollation
matrixfox
matrixfox

Reputation: 83

Combining CSV's from folder into one CSV in R

I'm attemping to write code that will take files from 'Inputs_Path' (Currently there are 6 .csv's there), combine them, and output them to 'Outputs_Path'

As of now my code will read the files in the folder just fine, but it breaks when it hits the 'else' part of my loop.

When I output the file, it outputs with only the first input file obviously. But I do know the output is working.

The error I'm getting is related to the files themselves.. like I'm pointing to the wrong place. Here is the error message;

Error in file(file, "rt") : cannot open the connection
In addition: Warning message:
In file(file, "rt") : cannot open file 'NA': No such file or directory

And here is the Code;

Inputs_Path <- "C:/MortgageSafe/Cirt2014-1/Sample"
Output_Path <- "C:/MortgageSafe/Cirt2014-1/BatchOutput"

setwd(Inputs_Path)

file_list<-list.files()
length_file_list<-length(file_list)

for(i in 1:length_file_list){

file_list<- as.character(file_list[i])
#do.call(rbind,lapply(file_list, read.csv, FALSE))
Mortgage_Sims<- read.table(file_list, header=TRUE, sep=",")

if(i == 1){
  Collated_Sims <- Mortgage_Sims
 }

else{
  Collated_Sims <- rbind(Collated_Sims, Mortgage_Sims)
  } 
}
Collated_Sims[, 1]<- noquote(Collated_Sims[, 1])

setwd(Output_Path)
write.table(Collated_Sims, "Collated Sims.csv", sep=",", row.names=FALSE, 
quote=FALSE)

Anyone have any suggestions on the else portion of it? Thanks in advance

Upvotes: 0

Views: 149

Answers (3)

ASH
ASH

Reputation: 20342

There are so many ways to merge all CSV files into a single DF.

setwd("C:/your_path_here")
fnames <- list.files()
csv <- lapply(fnames, read.csv)
result <- do.call(rbind, csv)

filedir <- setwd("C:/your_path_here")
file_names <- dir(filedir)
your_data_frame <- do.call(rbind,lapply(file_names,read.csv))

filedir <- setwd("C:/your_path_here")
file_names <- dir(filedir)
your_data_frame <- do.call(rbind, lapply(file_names, read.csv, skip = 1, header = FALSE))

filedir <- setwd("C:/your_path_here")
file_names <- dir(filedir)
your_data_frame <- do.call(rbind, lapply(file_names, read.csv, header = FALSE))

# 
temp <- setwd("C:/your_path_here")
temp = list.files(pattern="*.csv")
myfiles = lapply(temp, read.delim)

Upvotes: 0

Andrii
Andrii

Reputation: 3043

Here is the code for data loading if your csv files have the same structure 

# 1. Set path to the directory with data files
dir_data <- "..."

# 2. Set counter of loaded csv files
file_index <- 0

# 3. Read all csv files into one data set
for(file_name in list.files(dir_data)){

  # 3.1. Define path to csv file
  file_path <- paste0(dir_data, file_name)

  # 3.2. Load csv file
  df_file <- read.csv(file_path)

  # 3.3. Increment counter of loaded files
  file_index <- file_index + 1

  # 3.4. Merge data file with result data set 'df_result'
  if(file_index == 1){
    df_result <- df_file
  } else {
    df_result <- rbind(df_result, df_file)
 }

}

Upvotes: 0

Malte
Malte

Reputation: 36

file_list<- as.character(file_list[i])

You define the file list as a string here, in the following round of the loop you try to read from the second entry of the 'list' which is now a string and get NA as the result,

Rename file_list in the loop to file_name for example and it should work

file_name<- as.character(file_list[i]) 
Mortgage_Sims<-read.table(file_name,header=TRUE, sep=",")

Upvotes: 2

Related Questions