tibble::add_row to nested tibble throws error under tidyr 1.0.0

Question

In a former version of tidyr I was able to add rows to a nested tibble using tibble::add_row. After updating to version 1.0.0 I get the following error:

Error: levels.vctrs_list_of() not supported.

library(dplyr, warn.conflicts = FALSE)
library(tibble)
library(tidyr) # version 1.0.0

mtcars %>% 
  tidyr::nest(data = dplyr::select(., -cyl) %>% colnames) %>% 
  tibble::add_row(cyl = "all cyl", data = NA)
#> Error: `levels.vctrs_list_of()` not supported.

^{Created on 2020-01-17 by the reprex package (v0.3.0)}

And here is the same call using tidyr 0.8.3

library(dplyr, warn.conflicts = FALSE)
library(tibble)
library(tidyr) # version 0.8.3

 mtcars %>% 
   tidyr::nest(-cyl) %>% 
   tibble::add_row(cyl = "all cyl", data = NA)
  #> # A tibble: 4 x 2
  #>   cyl     data              
  #>   <chr>   <list>            
  #> 1 6       <tibble [7 × 10]> 
  #> 2 4       <tibble [11 × 10]>
  #> 3 8       <tibble [14 × 10]>
  #> 4 all cyl <lgl [1]>

^{Created on 2020-01-17 by the reprex package (v0.3.0)}

Are other users experiencing the same or is this specific to my system environment? Do I need to update other packages to get this running? Is there a workaround? Should I open an issue on Github?

Update: In the comments someone suggested updating the vctrs package. But after updating to the latest version 0.2.1 the same error still appears.

background / context

I should elaborate why I wanted to use add_row with data = NA in the first place.

My original script looked somewhat like this:

# tidyr version 0.8.3 add_row was working
iris %>% 
  tidyr::nest(-Species) %>% 
  tibble::add_row(Species = "All species", data = NA) %>% 
  mutate(data = purrr::modify_at(4, ~ as_tibble(select(iris, -Species)))

Which returned a nested tibble with all categories as well as an overall category.

Under tidyr 1.0.0 my code looks like this and throws said error:

# tidyr version 1.0.0 where add_row does not work
iris %>% 
  tidyr::nest(data = dplyr::select(., -Species) %>% colnames) %>% 
  tibble::add_row(Species = "All species", data = NA) %>% 
  mutate(data = purrr::modify_at(data, nrow(.), ~ as_tibble(select(iris, -Species))))

Since add_row is not working under tidyr 1.0.0 anymore, the next best similar alternative is as @IceCreamToucan suggested bind_rows.

# tidyr version 1.0.0 alternative with dpylr::bind_rows
iris %>% 
  tidyr::nest(data = dplyr::select(., -Species) %>% colnames) %>% 
  dplyr::bind_rows(tibble(Species = "All species", data = NA)) %>% 
  mutate(data = purrr::modify_at(data, nrow(.), ~ as_tibble(select(iris, -Species))))

However, since the syntax of tidyr::nest got more verbose under version 1.0.0 I tried to streamline the code and this seems by far the most straightforward approach, which I should have chosen from beginning on:

# What I should have been doing in the first place
iris %>% 
  dplyr::bind_rows(mutate(iris, Species = "All species")) %>% 
  tidyr::nest(data = dplyr::select(., -Species) %>% colnames) 

Answer 1

Following akrun's advice I installed the devel version of dplyr (and also tibble, tidyr, vctrs, tidyselect) and it seems that the next official tidyverse versions will solve this issue.

First of all, the original error message is now specific, and makes clear that adding a row without a common type is not possible.

library(tidyverse)
iris %>% 
  tidyr::nest(data = dplyr::select(., -Species) %>% colnames) %>% 
  tibble::add_row(tibble(Species = "All species", data = NA))
#> Error: No common type for `..1$data` <list> and `..2$data` <logical>.

^{Created on 2020-01-18 by the reprex package (v0.3.0)}

This behavior can be circumvented by wrapping NA in list(). Then we could use modify_at to replace the list(NA) with the original tibble.

library(tidyverse)
iris %>% 
   tidyr::nest(data = dplyr::select(., -Species) %>% colnames) %>% 
   tibble::add_row(Species = "All species", data = list(NA)) %>% 
   mutate(data = modify_at(data, nrow(.), ~ as_tibble(select(iris, -Species))))
#>  A tibble: 4 x 2
#>   Species     data              
#>   <chr>       <list>            
#> 1 setosa      <tibble [50 × 4]> 
#> 2 versicolor  <tibble [50 × 4]> 
#> 3 virginica   <tibble [50 × 4]> 
#> 4 All species <tibble [150 × 4]>

^{Created on 2020-01-18 by the reprex package (v0.3.0)}

However, now it is also possible to add tibbles as data argument with add_row by wrapping them in a list as in list(as_tibble(select(iris, -Species))):

library(tidyverse)
iris %>% 
  tidyr::nest(data = dplyr::select(., -Species) %>% colnames) %>% 
  tibble::add_row(Species = "All species",
                  data = list(as_tibble(select(iris, -Species))))
    #>  A tibble: 4 x 2
    #>   Species     data              
    #>   <chr>       <list>            
    #> 1 setosa      <tibble [50 × 4]> 
    #> 2 versicolor  <tibble [50 × 4]> 
    #> 3 virginica   <tibble [50 × 4]> 
    #> 4 All species <tibble [150 × 4]>

^{Created on 2020-01-18 by the reprex package (v0.3.0)}

Nevertheless, this approach is still more verbose then the one I came up with as an intermediate solution, which I will use from now on:

iris %>% 
  dplyr::bind_rows(mutate(iris, Species = "All species")) %>% 
  tidyr::nest(data = dplyr::select(., -Species) %>% colnames) 

Answer 2

There is a type issue and if we are appending NA elements in the list column, try

library(dplyr)
library(tidyr)
mtcars %>% 
  tidyr::nest(data = dplyr::select(., -cyl) %>% colnames) %>% 
  mutate(cyl = as.character(cyl)) %>%
  dplyr::add_row(cyl = "all cyl", 
    data = list(as.tibble(setNames(as.list(rep(NA_real_, ncol(mtcars) - 1)), 
         setdiff(names(mtcars), 'cyl'))))) 

Answer 3

A workaround is to use bind_rows. It looks like bind_rows produces a warning when you are adding to a list column, while add_row produces an error.

mtcars %>% 
  tidyr::nest(data = dplyr::select(., -cyl) %>% colnames) %>% 
  mutate_at(vars(cyl), as.character) %>% 
  bind_rows(tibble(cyl = 'all cyl', data = NA))

# # A tibble: 4 x 2
#   cyl     data              
#   <chr>   <list>            
# 1 6       <tibble [7 x 10]> 
# 2 4       <tibble [11 x 10]>
# 3 8       <tibble [14 x 10]>
# 4 all cyl <NULL>
# Warning message:
# In bind_rows_(x, .id) :
#   Vectorizing 'vctrs_list_of' elements may not preserve their attributes