CODEWITHSUNDEEP
rformula
Yabin Da
Yabin Da

Reputation: 595

How to handle string variable names in a nested formula in r

I am trying to run a loop of lm regressions with a list of variable names. Please see the code below. 

order <- c(4, 6, 8, 10, 12, 14, 16) # position of variables in my data frame

for(i in order) {
  m7 <- df %>% filter(names(df)[i] > 0) %>% 
    pdata.frame(index=c("index","Year")) %>% 
    plm(formula= log(names(df)[i]) ~ log(names(df)[i-1]) + 
          frez + log(dd) + I((log(dd))^2) + 
          preci + I(preci^2) + 
          + m7 + max.c + spei + lt + qt, 
        data=., effect = c("individual"),model = "within") %>% 
    coeftest(., vcov = function(x) vcovHC(x, method = 'arellano', cluster = 'group', type = 'HC3'))
}

I got an error message. I guess the problem might be with the log.

Error in log(names(df)[i]) : non-numeric argument to mathematical function

My variable names are

> names(df)
 [1] "index"           "Year"            "barley_harv"     "barley_yield"    "cotton_harv"     "cotton_yield"   
 [7] "oat_harv"        "oat_yield"       "peanut_harv"     "peanut_yield"    "rice_harv"       "rice_yield"     
[13] "sorghum_harv"    "sorghum_yield"   "sunflower_harv"  "sunflower_yield" "AOT40"           "frez"           
[19] "dd"              "spei"            "m7"              "max.c"           "preci"           "SUM60"          
[25] "W126"            "lt"              "qt"

Any ideas how should I fix the problem???

Upvotes: 0

Views: 87

Answers (3)

Anurag N. Sharma
Anurag N. Sharma

Reputation: 372

I'm guessing that the basic problem to address here is the Error in log(names(df)[i]) : non-numeric argument to mathematical function. This is because you aren't explicitly referring to the data when you frame your code as formula= log(names(df)[i]) ~ log(names(df)[i-1]) .... as explained below:

library(tidyverse)

# Dummy dataset
Data <- tibble(A = runif(100),
               B = runif(100))
# this works
lm(Data$B ~ Data$A) %>% broom::tidy()
# this works as well
lm(Data %>% pull(B) ~ Data %>% pull(A)) %>% broom::tidy()

# this will not work because you aren't referring to the variables within the lm() function
# but rather some random string that it does not understand
lm(names(Data)[2] ~ names(Data)[1], data = Data) %>% broom::tidy()

# A tidyeval solution for this
# Quoting your names
Name1 <- sym(names(Data)[2])
Name2 <- sym(names(Data)[1])
# Unquoting with the !! (bang bang) operator
lm(Data %>% pull(!!Name1) ~ Data %>% pull(!!Name2), data = Data) %>% broom::tidy()

The solution would be something as follows, although there is no sample data provided to create a full solution:

map(c(4,6,8,10,12,14,16),
    ~{
      # the first part of your code
      df <- df %>% 
        filter(names(df)[.x] > 0) %>% 
        pdata.frame(index=c("index","Year"))

      # the tidyeval solution
      Name1 <- sym(names(Data)[.x]) # Quoting
      Name2 <- sym(names(Data)[.x-1]) # Quoting

      df %>%
        plm(formula = log(pull(Data,!!Name1)) ~ log(pull(Data,!!Name2)) + # Unquoting
              frez + log(dd) + I((log(dd))^2) + preci + I(preci^2) + m7 + max.c + spei +lt + qt, 
            data= (.), 
            effect = c("individual"),
            model = "within") %>% 
        coeftest(., vcov = function(x) vcovHC(x, method = 'arellano', cluster = 'group', type = 'HC3'))
    }
)

Upvotes: 0

Yabin Da
Yabin Da

Reputation: 595

Problem solved with as.formula. See below. 

for(i in order) {

  m7.fl <- as.formula(paste(paste0('log(', names(df)[i], ')'), '~', paste0('log(', names(df)[i-1], ')'),'+',
                            'frez + log(dd) + I((log(dd))^2) + preci + I(preci^2) + m7 + max.c + spei + lt + qt', 
                      collapse = '+'))
  m7 <- df %>% filter(names(df)[i] > 0) %>% 
    pdata.frame(index=c("index","Year")) %>% 
    plm(formula= m7.fl, 
        data=., effect = c("individual"),model = "within") %>% 
    coeftest(., vcov = function(x) vcovHC(x, method = 'arellano', cluster = 'group', type = 'HC3'))
}

reformulate does the same trick. Either way one needs to specify the formula with paste. I was looking for something neater, i.e. directly use string variable names in formula.

Hopefully, other guys can provide more references on this matter.

Upvotes: 0

Ben Bolker
Ben Bolker

Reputation: 226732

Something like (code fragment):

## .. outside
other_vars <- c("frez","log(dd)", "I((log(dd))^2)", 
                "preci", "I(preci^2)", 
                "m7","max.c","spei","lt","qt")
## ... inside loop ...
ff <- reformulate(c(sprintf("log(%s)",names(df)[i-1]),other_vars),
                  response=sprintf("log(%s)",names(df)[i]))
m7 <- (df 
         %>% filter(names(df)[i] > 0) 
         %>% pdata.frame(index=c("index","Year")) 
         %>% plm(formula= ff, data=., effect = c("individual"),
            model = "within") 
         ## ... coeftest stuff here
      )

Upvotes: 1

Related Questions