Manipulate values within nested lists

Question

I am trying to read JSON metadata from an API, manipulate it in R, and then POST back.

However, when I GET the metadata through jsonlite, TRUE/FALSE values are read as logical and become upper case. The API will not accept uppercase TRUE/FALSE back. So I need to replace all TRUE/FALSE with "true" and "false" as characters, but retain the same nested list structure as the input.

The issue is that these are JSON key/value pairs and nested arrays, so the true/false values are nested at different levels. I also only want to change just the values which are "TRUE" or "FALSE", and not character strings that contain "true" or "false".

I have tried apply family of functions, purrr mapping, and recursive for loops.

#read data via API
dashitem_api<-paste0("api/metadata.json?filter=id:like:",dashitem_old_idprefix)
url<-paste0(baseurl,dashitem_api)

dash_items<-jsonlite::fromJSON(content(GET(url),"text")[])

... then replacements of IDs ....

EDIT: the replacement of IDs is what coerces the logical TRUE/FALSE values into character string "true"/"false".

Apologies for the verbose illustration...

    dashitem_old_idprefix<-"Ane0008"
    dashitem_new_idprefix<-"Ane0028"

    dash_items<-jsonlite::fromJSON(content(GET(url),"text")[])

    class(dash_items$charts$showData)
###output = "logical"

    #put all the replacement items into a list
    x1<-list(dashitem_old_idprefix,
             dashitem_new_idprefix)

    x2<-sapply(x1, function(x) as.character(x))

    replacements<-function(y){
      return(
        y %>% 
          gsub(x2[1], x2[2], .)
      )
    }

    new_dash <- rapply(dash_items, f = replacements, 
                       how = "replace")

    class(new_dash$charts$showData)
###output = "character"

The R object is a list, containing character vectors, named lists, unnamed lists, lists of character vectors, and data frames, something like this

new_dash<-list(charts=list(list(id="abcd123",shared="FALSE",translations=list(),
                 dimensions=data.frame(thisyear="FALSE",last6Months="TRUE"),
params=list(reportingPeriod="FALSE",reportingUnit="FALSE"),
dimensionItems=list(type="DATA_ELEMENT",dataElement=list(id="ZYXW987"))),

               list(id="abcd4567",shared="FALSE",translations=list(),
                dimensions=data.frame(thisyear="FALSE",last6Months="TRUE"),
                params=list(reportingPeriod="FALSE",reportingUnit="TRUE"),
dimensionItems=list(type="DATA_ELEMENT",dataElement=list(id="ZYXW988")))),

reportTables=list(id="abcd124",title="false positives", shared="FALSE",translations=list(),
 dimensions=data.frame(thisyear="FALSE",last6Months="TRUE"),
params=list(reportingPeriod="FALSE",reportingUnit="FALSE"),
dimensionItems=list(type="DATA_ELEMENT",dataElement=list(id="ZYXW989"))))

I found these solutions online, but I found I need to specify the name or location of the list with true/false data values, otherwise I get an error, or it changes the new_dash file in unwanted ways (adds a new nested list, for example).

#solution 1
change_list <- function(x) {
  for (i in seq_along(x)) {
    value <- x[[i]]
    if (is.list(value)) {
      x[[i]] <- change_list(value)
    } else {
      if (as.character(value)=="FALSE") {
        x[[i]] <- tolower(value)
      }
    }
  }
  x
}
test1<-change_list(new_dash)

#solution 2
test2<-lapply(new_dash, function(x) {
  id <- x == "FALSE"
  x[id] <- "false"
  return(x)
})


#solution 3
test3<- c(map(new_dash$charts, 
~modify_if(~x=="TRUE", tolower)), 
recursive= TRUE)

I probably need some purrr function that combines modify_if and modify_at. Or, an alternative way to read in the data that doesnt convert to logical TRUE/FALSE by default.

FWIW I'm an R newbie and would appreciate any answer no matter how complex or simple.

Answer 1

Are the values in your list object actually "TRUE" (R string) or TRUE (R logical)? If they're valid R logicals (unlike the example data you shared), then jsonlite::toJSON will correct them.

x <- list(
  partA = list(numbers = 1:3, boolean = T),
  partB = list(
    nested = list(
      numbers = 4:6, 
      nestB = list(boolean = c(FALSE, FALSE))
      )
    )
)

jsonlite::toJSON(x, pretty = T)

{
  "partA": {
    "numbers": [1, 2, 3],
    "boolean": [true]
  },
  "partB": {
    "nested": {
      "numbers": [4, 5, 6],
      "nestB": [
        {
          "boolean": false
        },
        {
          "boolean": false
        }
      ]
    }
  }
}

It seems unlikely that you generated strings of "TRUE" and "FALSE" in your data processing step (updated: that was actually the problem!), so hopefully this works. jsonlite::fromJSON converts [true, false] to c(TRUE, FALSE), and toJSON will do the inverse.

Making sure the dataframes are coerced to the same format may require some inspection. toJSON has some options: dataframe = can be

• "rows" (the default),
• "columns" "nestB": {"boolean": [false, false]}, or
• "values" "nestB": [[false], [false]]

But if you're using the defaults for reading the API response, you're unlikely to need to change the defaults for sending it back.

---

Updated:

This use-case was calling rapply to search over the whole list-object and replace certain elements. Because that was calling gsub, each element was being coerced to a character, including any numeric or logical values. To prevent this, you can use some_output_object <- rapply(some_input_object, f = some_replacing_function, how = "replace", classes = "character"). This leaves numeric and logical values unchanged, so that toJSON can correctly wrap them up.