R- How to read from multiple directories and apply function on same file names contained within different directories

Question

I have two directories that contain (share) files with the same name, e. g."file1","file2" and "file3", which can be found in dir1 as well as dir2. Now, I would like to read "file1" from dir1 and "file1" from dir2 in the first iteration and do some processing on both. In the second iteration, "dir1/file2" and "dir2/file2", and so forth.

I used the following code to read all files from ONE directory:

setwd("dir1")
file_list <- list.files()

process.Files <- function(file) {
myFile <- read.table(file, header=TRUE, sep="")
#process(myFile)
return(someDataFrame)
}

dataset <- do.call("cbind",lapply(file_list,
FUN=function(files){  process.Files(files)  }))

Now, I need something like:

file_list1 <- list.files("dir1/")
file_list2 <- list.files("dir2/")

compare.Files <- function(fileX,fileY) {
myFileX <- read.table(fileX, header=TRUE, sep="")
myFileY <- read.table(fileY, header=TRUE, sep="")
#compare(myFileX,myFileY)
return(someDataFrameWithComparisonResults)
}

#  I need to change this somehow to call compare.Files(files1,files2)
dataset <- do.call("cbind",lapply(file_list,
FUN=function(files){  process.Files(files)  }))

How could I make it work? Thanks in advance.

Answer 1

Are you looking for mapply which applies a function with multiple arguments to multiple vectors or lists?

mapply(compare.Files, file_list1, file_list2)

If you want to apply your function on all combinations of the files from both directories, you can use expand.grid. Let's make an example:

file_list1 = c("d1/a.txt", "d1/b.txt")
file_list2 = c("d2/a.txt", "d2/b.txt")
file.comb <- expand.grid(file_list1, file_list2)

and now apply compare.Files to both columns in your list:

result <- mapply(compare.Files, as.vector(file.comb[,1]), as.vector(file.comb[,2]))

Does this help?

Answer 2

I would be lazy and list all the files in one go and use regex to find the appropriate one for each iteration. Something along the lines of

# list all files with paths
(x <- list.files(full.names = TRUE, recursive = TRUE)) 

     [1] "./figure/delez_skupin.pdf"                                   "./figure/diag_efekt_odstrela.pdf"                           
     [3] "./figure/diag_maxent.pdf"                                    "./figure/diag_teza_v_casu.pdf"                              
     [5] "./figure/diag_teza_v_casu2.pdf"                              "./figure/efekt_odstrela.pdf"                                
     [7] "./figure/fig_teza.pdf"                                       "./figure/graf_odstrel_razmerje_kategorija.pdf"              
     [9] "./figure/graf_odstrel_razmerje_kategorija1.pdf"              "./figure/graf_odstrel_razmerje_kategorija2.pdf"             
    [11] "./figure/graf_starost_v_letih_skupaj.pdf"                    "./figure/korelacija_med_odstrelom_in_sist_1.pdf"            
    [13] "./figure/korelacija_med_odstrelom_in_sist_2.pdf"             "./figure/modeliranje_maxent_sistematicno.pdf"               
    [15] "./figure/plot_glm_maxent_model1.pdf"                         "./figure/plot_glm_maxent_model2.pdf"                        
    [17] "./figure/pregled_prostorskih_podatkov.pdf"                   "./figure/prikaz_okoljskih_spremenljivk1.pdf"                
    [19] "./figure/prikaz_okoljskih_spremenljivk2.pdf"                 "./figure/prikaz_okoljskih_spremenljivk3.pdf"                
    [21] "./figure/prikaz_okoljskih_spremenljivk4.pdf"                 "./figure/priloznostna_glede_na_mesec.pdf"                   
    [23] "./figure/primerjava_spremenljivk_glede_prisotnosti.pdf"      "./figure/priprava_primerjava.pdf"                           
    [25] "./figure/razsirjenost_gamsa_tnp.pdf"                         "./figure/razsirjenost_gamsa_v_tnp.pdf"                      
    [27] "./figure/sprememba_strukture_po_mesecih.pdf"                 "./figure/sprememba_strukture_po_mesecih_abs.pdf"            
    [29] "./figure/sprememba_strukture_po_mesecih_rel.pdf"             "./figure/st_osebkov_na_leto_priloznostna.pdf"               
    [31] "./figure/st_osebkov_na_leto_sistematicna.pdf"                "./figure/teza_enoletnikov.pdf"                              
    [33] "./figure/vpliv_js_glm1.pdf"                                  "./figure/vpliv_js_glm2.pdf"                                 
...                                   
    [51] "./ostale_slike/naslovnica_gams.jpg"                          "./ostale_slike/nepipaj/naslovnica_gams.jpg"                 
    [53] "./ostale_slike/nepipaj/slika17_odlov_tone.jpg"               "./ostale_slike/nepipaj/slika18_odlov_irena.jpg"             
    [55] "./ostale_slike/nepipaj/slika19_odlov_irena_markica.jpg"      "./ostale_slike/nepipaj/slika20_odlov_luna.jpg"              
    [57] "./ostale_slike/nepipaj/slika21_gibanje_irena.png"            "./ostale_slike/nepipaj/slika22_gibanje_mojca.png"           
    [59] "./ostale_slike/nepipaj/slika23_gibanje_tone.png"             "./ostale_slike/nepipaj/slika24_gibanje_luna.png"            
    [61] "./ostale_slike/nepipaj/slika25_gibanje_irena_jesen_zima.png" "./ostale_slike/nepipaj/slika26_gibanje_mojca_jesen_zima.png"
    [63] "./ostale_slike/nepipaj/slika27_gibanje_tone_jesen_zima.png"  "./ostale_slike/nepipaj/slika28_graf_aktivnosti.jpg"         
    [65] "./ostale_slike/razsirjenost_gamsa_slovenija.png"             "./ostale_slike/slika17_odlov_tone.jpg"                      
    [67] "./ostale_slike/slika18_odlov_irena.jpg"                      "./ostale_slike/slika19_odlov_irena_markica.jpg"             
    [69] "./ostale_slike/slika20_odlov_luna.jpg"                       "./ostale_slike/slika21_gibanje_irena.jpg"                   
    [71] "./ostale_slike/slika22_gibanje_mojca.jpg"                    "./ostale_slike/slika23_gibanje_tone.jpg"                    
    [73] "./ostale_slike/slika24_gibanje_luna.jpg"                     "./ostale_slike/slika25_gibanje_irena_jesen_zima.jpg"        
    [75] "./ostale_slike/slika26_gibanje_mojca_jesen_zima.jpg"         "./ostale_slike/slika27_gibanje_tone_jesen_zima.jpg"         
    [77] "./ostale_slike/slika28_graf_aktivnosti.jpg"                  "./ostale_slike/slo_gams.bmp" 

# find all files that start with "slika2"
x[grepl("slika2", x)]
 [1] "./ostale_slike/nepipaj/slika20_odlov_luna.jpg"               "./ostale_slike/nepipaj/slika21_gibanje_irena.png"           
 [3] "./ostale_slike/nepipaj/slika22_gibanje_mojca.png"            "./ostale_slike/nepipaj/slika23_gibanje_tone.png"            
 [5] "./ostale_slike/nepipaj/slika24_gibanje_luna.png"             "./ostale_slike/nepipaj/slika25_gibanje_irena_jesen_zima.png"
 [7] "./ostale_slike/nepipaj/slika26_gibanje_mojca_jesen_zima.png" "./ostale_slike/nepipaj/slika27_gibanje_tone_jesen_zima.png" 
 [9] "./ostale_slike/nepipaj/slika28_graf_aktivnosti.jpg"          "./ostale_slike/slika20_odlov_luna.jpg"                      
[11] "./ostale_slike/slika21_gibanje_irena.jpg"                    "./ostale_slike/slika22_gibanje_mojca.jpg"                   
[13] "./ostale_slike/slika23_gibanje_tone.jpg"                     "./ostale_slike/slika24_gibanje_luna.jpg"                    
[15] "./ostale_slike/slika25_gibanje_irena_jesen_zima.jpg"         "./ostale_slike/slika26_gibanje_mojca_jesen_zima.jpg"        
[17] "./ostale_slike/slika27_gibanje_tone_jesen_zima.jpg"          "./ostale_slike/slika28_graf_aktivnosti.jpg"

Having full file names you can import your data sets and manipulate them further.