CODEWITHSUNDEEP
rnullnadata-cleaning
Sammy
Sammy

Reputation: 15

NA to replace NULL in list/for loop

I am trying to replace NULL values with NAs in a list pulled from an API, but the lengths are different and therefore can't be replaced.

I have tried using the nullToNA function in the toxboot package (found here), but it won't locate the function in R when I try to call it (I don't know if there have been changes to the package which I can't locate or whether it is because the list is not pulled from a MongoDB). I have also tried all the function call checks here . My code is below. Any help?

library(httr)
library(toxboot)
library(RJSONIO)
library(lubridate)
library(xlsx)
library(reshape2)

resUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3010CO3.M"

comUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3020CO3.M"

indUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3035CO3.M"

apiList <- list(resUrl, comUrl, indUrl)

results <- vector("list", length(apiList))

for(i in length(apiList)){
  raw <- GET(url = as.character(apiList[i]))
  char <- rawToChar(raw$content)
  list <- fromJSON(char)
    for (j in length(list$series[[1]]$data)){
      if (is.null(list$series[[1]]$data[[j]][[2]])== TRUE)
        ##nullToNA(list$series[[1]]$data[[j]][[2]])
        ##list$series[1]$data[[j]][[2]] <- NA
      else
        next
    }
  ##seriesData <- list$series[[1]]$data
  unlistResult <- lapply(list, unlist)
  ##unlistResult <- lapply(seriesData, unlist)
  ##unlist2 <- lapply(unlistResult,unlist)
  ##results[[i]] <- unlistResult
  results[[i]] <- unlistResult
}

My hashtags have some of the things that I have tried. But there are a few other methods I haven't tried.

I have seen lapply(list, function(x) ifelse (x == "NULL", NA, x)) but haven't had any luck with that eiter.

Upvotes: 1

Views: 1203

Answers (1)

r2evans
r2evans

Reputation: 161155

Try this:

library(httr)
resUrl <- "http://api.eia.gov/series/?api_key=2B5239FA427673D22505DBF45664B12E&series_id=NG.N3010CO3.M"
x <- GET(resUrl)
y <- content(x)
str(head(y$series[[1]]$data))
# List of 6
#  $ :List of 2
#   ..$ : chr "201701"
#   ..$ : NULL
#  $ :List of 2
#   ..$ : chr "201612"
#   ..$ : num 6.48
#  $ :List of 2
#   ..$ : chr "201611"
#   ..$ : num 7.42
#  $ :List of 2
#   ..$ : chr "201610"
#   ..$ : num 9.75
#  $ :List of 2
#   ..$ : chr "201609"
#   ..$ : num 12.1
#  $ :List of 2
#   ..$ : chr "201608"
#   ..$ : num 14.3

In this first URL, only the first within $series[[1]]$data contained a NULL. BTW: be clear to distinguish between NULL (the literal) and "NULL" (a character string with 4 letters).

Here are some ways (with various data types) to check for NULLs:

is.null(NULL)
# [1] TRUE
length(NULL)
# [1] 0

Simple enough so far, let's try to list with NULLs:

l <- list(NULL, 1)
is.null(l)
# [1] FALSE
sapply(l, is.null)
# [1]  TRUE FALSE
length(l)
# [1] 2
lengths(l)
# [1] 0 1
sapply(l, length)
# [1] 0 1

(The "0" lengths indicate NULLs.) I'll use lengths here:

y$series[[1]]$data <- lapply(y$series[[1]]$data, function(z) { z[ lengths(z) == 0 ] <- NA; z; })
str(head(y$series[[1]]$data))
# List of 6
#  $ :List of 2
#   ..$ : chr "201701"
#   ..$ : logi NA
#  $ :List of 2
#   ..$ : chr "201612"
#   ..$ : num 6.48
#  $ :List of 2
#   ..$ : chr "201611"
#   ..$ : num 7.42
#  $ :List of 2
#   ..$ : chr "201610"
#   ..$ : num 9.75
#  $ :List of 2
#   ..$ : chr "201609"
#   ..$ : num 12.1
#  $ :List of 2
#   ..$ : chr "201608"
#   ..$ : num 14.3

Upvotes: 3

Related Questions