R: Subset data frames in a list by length of data frames in another list

Question

I have two lists of data frames. Both of same length, same order, and identical names for each data frame.

df1 <- data.frame(
  Code = c("A1","A1","A1","B1","B1","B1","C1","C1","C1"),
  Name = c("Cat", "Dog", "Horse","Cat", "Dog", "Horse","Cat", "Dog", "Horse"),
  freq = c(4,5,6,4,2,6,8,1,5)
)

list1 <- split(df1, df1$Code)


df2 <- data.frame(
  Code = c("A1","A1","A1","B1","B1","B1","C1","C1","C1"),
  filler = c(2,8,4,1,6,9,4,1,2),
  filler1 = c(4,5,6,4,2,6,8,1,5)
)

list2 <- split(df2, df2$Code)

I want to subset all data frames in list1. I want to remove all rows where the value of 'freq' is less than or equal to the number of rows in the data frame of same name in list2.

The desired result is as follows:

df3 <- data.frame(
  Code = c("A1","A1","A1","B1","B1","C1","C1"),
  Name = c("Cat", "Dog", "Horse","Cat", "Horse","Cat", "Horse"),
  freq = c(4,5,6,4,6,8,5)
)
 
list3 <- split(df3, df3$Code)

I have tried the following

list3 <- lapply(list1, function(x) x[x$freq>nrow(list2$x),])

Unfortunately it seems lapply can't iterate over a second list even though the elements have identical names. I have tried mapply but failed to get it to run at all.

Answer 1

If you start from list1 and List2

l <- sapply(list2, nrow)
lapply(list1, \(x) subset(x, freq > l[Code]))

gives

$A1
  Code  Name freq
1   A1   Cat    4
2   A1   Dog    5
3   A1 Horse    6

$B1
  Code  Name freq
4   B1   Cat    4
6   B1 Horse    6

$C1
  Code  Name freq
7   C1   Cat    8
9   C1 Horse    5

---

If you start from df1 and df2

Actually you can bypass the steps of yielding list1 and list2, but using df1 and df2 directly. You can try the following options:

• Base r approach: split + subset + table

> split(subset(df1, freq > table(df2$Code)[Code]), ~Code)
$A1
  Code  Name freq
1   A1   Cat    4
2   A1   Dog    5
3   A1 Horse    6

$B1
  Code  Name freq
4   B1   Cat    4
6   B1 Horse    6

$C1
  Code  Name freq
7   C1   Cat    8
9   C1 Horse    5

• dplyr approach: left_join + filter + group_split

library(dplyr)

df1 %>%
    left_join(count(df2, Code)) %>%
    filter(freq > n) %>%
    select(-n) %>%
    group_split(Code)

which gives

[[1]]
# A tibble: 3 × 3
  Code  Name   freq
  <chr> <chr> <dbl>
1 A1    Cat       4
2 A1    Dog       5
3 A1    Horse     6

[[2]]
# A tibble: 2 × 3
  Code  Name   freq
  <chr> <chr> <dbl>
1 B1    Cat       4
2 B1    Horse     6

[[3]]
# A tibble: 2 × 3
  Code  Name   freq
  <chr> <chr> <dbl>
1 C1    Cat       8
2 C1    Horse     5

Answer 2

Using lapply

# Subset list1 based on the condition
filtered_list1 <- lapply(names(list1), function(nm) {
  df <- list1[[nm]]  # Extract the dataframe from list1
  threshold <- nrow(list2[[nm]])  # Get the number of rows in the corresponding dataframe from list2
  df[df$freq > threshold, ]  # Keep only rows where freq is greater than the threshold
})

# Convert list back to named list
names(filtered_list1) <- names(list1)

# Print the result
filtered_list1

Answer 3

lapply is best used to iterate items in a single list. To work on parallel items within two or more lists, use Map instead.

Map(function(a, b) a[a$freq > nrow(b),], list1, list2)
#> $A1
#>   Code  Name freq
#> 1   A1   Cat    4
#> 2   A1   Dog    5
#> 3   A1 Horse    6
#> 
#> $B1
#>   Code  Name freq
#> 4   B1   Cat    4
#> 6   B1 Horse    6
#> 
#> $C1
#>   Code  Name freq
#> 7   C1   Cat    8
#> 9   C1 Horse    5

If you want to use lapply, use the names of list1 to access each member of list1 and list2 inside your function:

names(list1) |>
  lapply(function(x) list1[[x]][list1[[x]]$freq > nrow(list2[[x]]),]) |>
  setNames(names(list1))
#> $A1
#>   Code  Name freq
#> 1   A1   Cat    4
#> 2   A1   Dog    5
#> 3   A1 Horse    6
#> 
#> $B1
#>   Code  Name freq
#> 4   B1   Cat    4
#> 6   B1 Horse    6
#> 
#> $C1
#>   Code  Name freq
#> 7   C1   Cat    8
#> 9   C1 Horse    5