Fixing a multiple warning "unknown column"

Question

I have a persistent multiple warning of "unknown column" for all types of commands (e.g., str(x) to installing updates on packages), and not sure how to debug this or fix it.

The warning "unknown column" is clearly related to a variable in a tbl_df that I renamed, but the warning comes up in all kinds of commands seemingly unrelated to the tbl_df (e.g., installing updates on a package, str(x) where x is simply a character vector).

Answer 1

Building on the answer by @stok ( https://stackoverflow.com/a/47848259/7733418 ), who found this problem when using group_by (which also converts your data.frame to a tibble), and solved it in the same way.

For me the problem was ultimately due to the use of "slice()". Slice() converted my data.frame to a tibble, causing this error.

Checking the class of your data.frame and re-converting it to a data.frame whenever a function converts it to a tibble could solve this issue.

Answer 2

I know this is an old thread, but I just encountered the same problem when loading a spatial vector in geopackage format with the package sf. Using as_tibble=FALSE worked for me. The file was loaded as an sp object but everything still worked fine. As mentioned by @sabre, trying to force an object into a tibble seems to be making the problems while trying to index a column that was not anymore there.

Answer 3

This is an issue with the Diagnostics tool in RStudio (the tool that shows warnings and possible mistakes in your code). It was partially fixed at this commit in RStudio v1.1.103 or later by @kevin-ushey. That fix was partial, because the warnings still appeared (albeit with less frequency). This issue was reported with a reproducible example at https://github.com/rstudio/rstudio/issues/7372 and it was fixed on RStudio v1.4 pull request.

Update to the latest RStudio release to fix this issue. Alternatively, there are several workarounds available, choose the solution you prefer:

• Disable the code diagnostics for all files in Preferences/Code/Diagnostics

• Disable all diagnostics for a specific file:

  Add at the beginning of the opened file(s):

   # !diagnostics off
  

  Then save the files and the warnings should stop appearing.

• Disable the diagnostics for the variables that cause the warning

  Add at the beginning of the opened file(s):

   # !diagnostics suppress=<comma-separated list of variables>
  

  Then save the files and the warnings should stop appearing.

The warnings appear because the diagnostics tool in RStudio parses the source code to detect errors and when it performs the diagnostic checks it accesses columns in your tibble that are not initialized, giving the Warning we see. The warnings do not appear because you run unrelated things, they appear when the RStudio diagnostics are executed (when a file is saved, then modified, when you run something...).

Answer 4

I get these warnings when I rename a column using dplyr::rename after reading it using the readr package.

The old name of the column is not renamed in the spec attribute. So removing the the spec attribute makes the warnings go away. Also removing the "spec_tbl_df" class seems like a good idea.

attr(dat, "spec") <- NULL
class(dat) <- setdiff(class(dat), "spec_tbl_df")

Answer 5

I had this problem when dealing with tibble and lapply functions together. The tibble seemed to save things as a list inside the dataframe.

I solved it by using unlist before adding the results of an lapply function to the tibble.

Answer 6

Converting the class into data.frame solved the problem for me:

library(dplyr)
df <- data.frame(id = c(1,1:3), name = c("mary", "jo", "jill","steve"))
dfTbl <- df %>%
  group_by(id) %>%
  summarize (n = n())
class(dfTbl) # [1] "tbl_df"     "tbl"        "data.frame"
dfTbl = as.data.frame(dfTbl)
class(dfTbl) # [1] "data.frame"

Borrowed the partial script from @adts

Answer 7

I have faced this issue when using the "dplyr" package.
For those facing this problem after using the "group_by" function in the "dplyr" library:

I have found that ungrouping the variables solves the unknown column warning problem. Sometimes I have had to iterate through the ungrouping several times until the problem is resolved.

Answer 8

Let's say I wanted to select the following column(s)

best.columns = 'id'

For me the following gave the warning:

df%>% select_(one_of(best.columns))

While this worked as expected, although, as far as I know dplyr, this should be identical.

df%>% select_(.dots = best.columns)

Answer 9

I ran into this problem too except through a tibble created using a dyplyr block. Here's slight modification of sabre's code to show how I came to the same error.

library(dplyr)

df <- data.frame(id = c(1,1:3), name = c("mary", "jo", "jill","steve"))

t <- df %>%
  group_by(id) %>%
  summarize (n = n())

t
str(t)


t$newvar[t$id==1] <- 0

Answer 10

I have been encountering the same problem, and although I don't know why it occurs, I have been able to pin down when it occurs, and thus prevent it from happening.

The issue seems to be with adding in a new column, derived from indexing, in a base R data frame vs. in a tibble data frame. Take this example, where you add a new column (age) to a base R data frame:

base_df <- data.frame(id = c(1:3), name = c("mary", "jill","steve"))

base_df$age[base_df$name == "mary"] <- 47

That works without returning a warning. But when the same is done with a tibble, it throws a warning (and consequently, I think causing the weird, seemingly unprovoked, multiple warning issue):

library(tibble)

tibble_df <- tibble(id = c(1:3), name = c("mary", "jill","steve"))

tibble_df$age[tibble_df$name == "mary"] <- 47

Warning message:
Unknown column 'age' 

There are surely better ways of avoiding this, but I have found that first creating a vector of NAs does the job:

tibble_df$age <- NA

tibble_df$age[tibble_df$name == "mary"] <- 47