CODEWITHSUNDEEP
rlistdata.table
Pierre Lapointe
Pierre Lapointe

Reputation: 16277

lapply with nested list

I have a nested list and I'd like to lapply as.data.frame on the deepest nesting level and then rbindlist (from data.table) everything. Here's what my data looks like:

a <- list(date="2017-01-01",ret=1:5)
b <- list(date="2017-01-02",ret=7:9)
lvl3 <- list(a,b) 
lvl2 <- list(lvl3,lvl3)
lvl1 <- list(lvl2,lvl2,lvl2)

If I only had lvl3, I would to this to tranform into a data.frame and rbind the data:

rbindlist(lapply(lvl3,as.data.frame))
         date ret
1: 2017-01-01   1
2: 2017-01-01   2
3: 2017-01-01   3
4: 2017-01-01   4
5: 2017-01-01   5
6: 2017-01-02   7
7: 2017-01-02   8
8: 2017-01-02   9

How would I do that from lvl1 and rbind all nested data.frames? This does not work:

rbindlist(lapply(lvl1,as.data.frame))

Desired result contains 48 rows:

         date ret
 1: 2017-01-01   1
 2: 2017-01-01   2
 3: 2017-01-01   3
 4: 2017-01-01   4
 5: 2017-01-01   5
 6: 2017-01-02   7
 7: 2017-01-02   8
 8: 2017-01-02   9
 9: 2017-01-01   1
10: 2017-01-01   2
11: 2017-01-01   3
12: 2017-01-01   4
13: 2017-01-01   5
14: 2017-01-02   7
15: 2017-01-02   8
16: 2017-01-02   9
17: 2017-01-01   1
18: 2017-01-01   2
19: 2017-01-01   3
20: 2017-01-01   4
21: 2017-01-01   5
22: 2017-01-02   7
23: 2017-01-02   8
24: 2017-01-02   9
25: 2017-01-01   1
26: 2017-01-01   2
27: 2017-01-01   3
28: 2017-01-01   4
29: 2017-01-01   5
30: 2017-01-02   7
31: 2017-01-02   8
32: 2017-01-02   9
33: 2017-01-01   1
34: 2017-01-01   2
35: 2017-01-01   3
36: 2017-01-01   4
37: 2017-01-01   5
38: 2017-01-02   7
39: 2017-01-02   8
40: 2017-01-02   9
41: 2017-01-01   1
42: 2017-01-01   2
43: 2017-01-01   3
44: 2017-01-01   4
45: 2017-01-01   5
46: 2017-01-02   7
47: 2017-01-02   8
48: 2017-01-02   9

Upvotes: 4

Views: 1705

Answers (5)

Frank
Frank

Reputation: 66819

@docendo's general solution is best, in my opinion, but if you know that it is only nested two-deep...

library(magrittr)

lvl1 %>% 
  unlist(recursive=FALSE) %>% 
  unlist(recursive=FALSE) %>% 
  lapply(as.data.table) %>% 
  rbindlist

From @lmo, here's the pipeless analogue (that doesn't require magrittr):

do.call(
  rbind, 
  lapply(
    unlist(unlist(lvl1, recursive=FALSE), recursive=FALSE), 
    as.data.frame
  )
)

Upvotes: 3

amonk
amonk

Reputation: 1795

I would go with the wicked package purrr. In particular: 

library(purrr)

(rbindlist(lapply(simplify_all((rbindlist((lvl1 %>% at_depth(3,data.frame))))),rbindlist)))

     date ret
1: 2017-01-01   1
2: 2017-01-01   2
3: 2017-01-01   3
4: 2017-01-01   4
5: 2017-01-01   5
-----
44: 2017-01-01   4
45: 2017-01-01   5
46: 2017-01-02   7
47: 2017-01-02   8
48: 2017-01-02   9

Upvotes: 1

talat
talat

Reputation: 70266

You can build your own recursive function, à la

f <- function(l) {
  data.table::rbindlist(lapply(l, function(x) {
    if(all(sapply(x, is.atomic))) as.data.table(x) else f(x)
  }))
}
f(lvl1)

This returns an ordinary data.table of 48 rows and 2 columns.

Also note that this works with lvl1, lvl2, and lvl3 without modifications.

Upvotes: 5

Val
Val

Reputation: 7023

An ugly nested lapply call with do.call will do the trick:

do.call(rbind,do.call(rbind,lapply(lvl1,function(x) lapply(x,function(y) do.call(rbind,lapply(y, function(z) as.data.frame(z)))))))

The output:

> do.call(rbind,do.call(rbind,lapply(lvl1,function(x) lapply(x,function(y) do.call(rbind,lapply(y, function(z) as.data.frame(z)))))))
         date ret
1  2017-01-01   1
2  2017-01-01   2
3  2017-01-01   3
4  2017-01-01   4
5  2017-01-01   5
6  2017-01-02   7
7  2017-01-02   8
8  2017-01-02   9
9  2017-01-01   1
10 2017-01-01   2
11 2017-01-01   3
12 2017-01-01   4
13 2017-01-01   5
14 2017-01-02   7
15 2017-01-02   8
16 2017-01-02   9
17 2017-01-01   1
18 2017-01-01   2
19 2017-01-01   3
20 2017-01-01   4
21 2017-01-01   5
22 2017-01-02   7
23 2017-01-02   8
24 2017-01-02   9
25 2017-01-01   1
26 2017-01-01   2
27 2017-01-01   3
28 2017-01-01   4
29 2017-01-01   5
30 2017-01-02   7
31 2017-01-02   8
32 2017-01-02   9
33 2017-01-01   1
34 2017-01-01   2
35 2017-01-01   3
36 2017-01-01   4
37 2017-01-01   5
38 2017-01-02   7
39 2017-01-02   8
40 2017-01-02   9
41 2017-01-01   1
42 2017-01-01   2
43 2017-01-01   3
44 2017-01-01   4
45 2017-01-01   5
46 2017-01-02   7
47 2017-01-02   8
48 2017-01-02   9

Upvotes: 1

caw5cv
caw5cv

Reputation: 721

There's probably more elegant ways, but combining data.table with nested foreach loops:

library(foreach)
library(data.table)

a <- list(date="2017-01-01",ret=1:5)
b <- list(date="2017-01-02",ret=7:9)
lvl3 <- list(a,b) 
lvl2 <- list(lvl3,lvl3)
lvl1 <- list(lvl2,lvl2,lvl2)

o.3 <- foreach(i=1:length(lvl1)) %do% {
    o.2 <- foreach(j=1:length(lvl1[[i]])) %do% {
            o.1 <- foreach(k=1:length(lvl1[[i]][[j]])) %do% {
                as.data.table(lvl1[[i]][[j]][[k]])
            }
            rbindlist(o.1)
        }
        rbindlist(o.2)
    }

dat.final <- rbindlist(o.3)

Upvotes: 2

Related Questions