CODEWITHSUNDEEP
rlistnested-lists
pssguy
pssguy

Reputation: 3515

How can I create a data.frame from a nested list with differing numbers of variable

I have downloaded a json file of Nobel Laureates and transformed it to a list, named 'nobels'. A couple of records are here shown in the structure 

str(nobels)  

List of 1
$ laureates:List of 2
  ..$ :List of 13
  .. ..$ id             : chr "359"
  .. ..$ firstname      : chr "Axel Hugo Theodor"
  .. ..$ surname        : chr "Theorell"
  .. ..$ born           : chr "1903-07-06"
  .. ..$ died           : chr "1982-08-15"
  .. ..$ bornCountry    : chr "Sweden"
  .. ..$ bornCountryCode: chr "SE"
  .. ..$ bornCity       : chr "Linköping"
  .. ..$ diedCountry    : chr "Sweden"
  .. ..$ diedCountryCode: chr "SE"
  .. ..$ diedCity       : chr "Stockholm"
  .. ..$ gender         : chr "male"
  .. ..$ prizes         :List of 1
  .. .. ..$ :List of 5
  .. .. .. ..$ year        : chr "1955"
  .. .. .. ..$ category    : chr "medicine"
  .. .. .. ..$ share       : chr "1"
  .. .. .. ..$ motivation  : chr "\"for his discoveries concerning the nature and mode of action of oxidation enzymes\""
  .. .. .. ..$ affiliations:List of 1
  .. .. .. .. ..$ :List of 3
  .. .. .. .. .. ..$ name   : chr "Karolinska Institutet, Nobel Medical Institute"
  .. .. .. .. .. ..$ city   : chr "Stockholm"
  .. .. .. .. .. ..$ country: chr "Sweden"

  ..$ :List of 10
  .. ..$ id             : chr "774"
  .. ..$ firstname      : chr "Richard"
  .. ..$ surname        : chr "Axel"
  .. ..$ born           : chr "1946-07-02"
  .. ..$ died           : chr "0000-00-00"
  .. ..$ bornCountry    : chr "USA"
  .. ..$ bornCountryCode: chr "US"
  .. ..$ bornCity       : chr "New York, NY"
  .. ..$ gender         : chr "male"
  .. ..$ prizes         :List of 1
  .. .. ..$ :List of 5
  .. .. .. ..$ year        : chr "2004"
  .. .. .. ..$ category    : chr "medicine"
  .. .. .. ..$ share       : chr "2"
  .. .. .. ..$ motivation  : chr "\"for their discoveries of odorant receptors and the organization of the olfactory system\""
  .. .. .. ..$ affiliations:List of 1
  .. .. .. .. ..$ :List of 3
  .. .. .. .. .. ..$ name   : chr "Columbia University"
  .. .. .. .. .. ..$ city   : chr "New York, NY"
  .. .. .. .. .. ..$ country: chr "USA"

How should I go about transforming this into a data.frame?

Although there are lists within lists, I am happy to use, say, year and category and dispense with prizes.

There is also the problem that not every record has the same number of variables - the second example here, for example, not providing the diedCountry field, amongst others

TIA

Profuse apologies. I should not really be doing this at night. The answers provided are fine for my original question. However, when I run the full list, I get an error

Error in data.frame(year = "1931", category = "literature", share = "1",  : 
arguments imply differing number of rows: 1, 0

Below is data causing this. It looks like it has something to do with the affiliations

nobels <- list(structure(list(id = "359", firstname = "Axel Hugo Theodor", 
surname = "Theorell", born = "1903-07-06", died = "1982-08-15", 
bornCountry = "Sweden", bornCountryCode = "SE", bornCity = "Linköping", 
diedCountry = "Sweden", diedCountryCode = "SE", diedCity = "Stockholm", 
gender = "male", prizes = list(structure(list(year = "1955", 
category = "medicine", share = "1", motivation = "\"for his discoveries concerning the          nature and mode of action of oxidation enzymes\"", 
affiliations = list(structure(list(name = "Karolinska Institutet, Nobel Medical    Institute", 
city = "Stockholm", country = "Sweden"), .Names = c("name", 
"city", "country")))), .Names = c("year", "category", 
"share", "motivation", "affiliations")))), .Names = c("id", 
"firstname", "surname", "born", "died", "bornCountry", "bornCountryCode", 
"bornCity", "diedCountry", "diedCountryCode", "diedCity", "gender", 
"prizes")), structure(list(id = "604", firstname = "Erik Axel", 
surname = "Karlfeldt", born = "1864-07-20", died = "1931-04-08", 
bornCountry = "Sweden", bornCountryCode = "SE", bornCity = "Karlbo", 
diedCountry = "Sweden", diedCountryCode = "SE", diedCity = "Stockholm", 
gender = "male", prizes = list(structure(list(year = "1931", 
category = "literature", share = "1", motivation = "\"The poetry of Erik Axel   Karlfeldt\"", 
affiliations = list(list())), .Names = c("year", "category", 
"share", "motivation", "affiliations")))), .Names = c("id", 
"firstname", "surname", "born", "died", "bornCountry", "bornCountryCode", 
"bornCity", "diedCountry", "diedCountryCode", "diedCity", "gender", 
"prizes")))

Upvotes: 0

Views: 127

Answers (2)

akrun
akrun

Reputation: 887951

You could also use unnest from tidyr

 devtools::install_github("hadley/tidyr")
 library(tidyr)

Using your new dataset, this seems to work

  res1 <-unnest(lapply(nobels, function(x)
         as.data.frame.list(rapply(x,unlist), stringsAsFactors=FALSE)))

  str(res1)
  #Classes ‘tbl_df’, ‘tbl’ and 'data.frame':    2 obs. of  19 variables:
  #   $ id                         : chr  "359" "604"
  #$ firstname                  : chr  "Axel Hugo Theodor" "Erik Axel"
  #$ surname                    : chr  "Theorell" "Karlfeldt"
  #$ born                       : chr  "1903-07-06" "1864-07-20"
  #$ died                       : chr  "1982-08-15" "1931-04-08"
  #$ bornCountry                : chr  "Sweden" "Sweden"
  #$ bornCountryCode            : chr  "SE" "SE"
  #$ bornCity                   : chr  "Linköping" "Karlbo"
  #$ diedCountry                : chr  "Sweden" "Sweden"
  #$ diedCountryCode            : chr  "SE" "SE"
  #$ diedCity                   : chr  "Stockholm" "Stockholm"
  #$ gender                     : chr  "male" "male"
  #$ prizes.year                : chr  "1955" "1931"
  #$ prizes.category            : chr  "medicine" "literature"
  #$ prizes.share               : chr  "1" "1"
  #$ prizes.motivation          : chr  "\"for his discoveries concerning the          nature and mode of action of oxidation enzymes\"" "\"The poetry of Erik Axel   Karlfeldt\""
  #$ prizes.affiliations.name   : chr  "Karolinska Institutet, Nobel Medical    Institute" NA
 #$ prizes.affiliations.city   : chr  "Stockholm" NA
 #$ prizes.affiliations.country: chr  "Sweden" NA

Upvotes: 1

Ujjwal
Ujjwal

Reputation: 3168

As you identified correctly, the problem is occurring due to affiliations, whose sub-list is an empty list. 

> str(nobels)
List of 2
 $ :List of 13
  ..$ id             : chr "359"
  ..$ firstname      : chr "Axel Hugo Theodor"
  ..$ surname        : chr "Theorell"
  ..$ born           : chr "1903-07-06"
  ..$ died           : chr "1982-08-15"
  ..$ bornCountry    : chr "Sweden"
  ..$ bornCountryCode: chr "SE"
  ..$ bornCity       : chr "Linköping"
  ..$ diedCountry    : chr "Sweden"
  ..$ diedCountryCode: chr "SE"
  ..$ diedCity       : chr "Stockholm"
  ..$ gender         : chr "male"
  ..$ prizes         :List of 1
  .. ..$ :List of 5
  .. .. ..$ year        : chr "1955"
  .. .. ..$ category    : chr "medicine"
  .. .. ..$ share       : chr "1"
  .. .. ..$ motivation  : chr "\"for his discoveries concerning the          nature and mode of action of oxidation enzymes\""
  .. .. ..$ affiliations:List of 1
  .. .. .. ..$ :List of 3
  .. .. .. .. ..$ name   : chr "Karolinska Institutet, Nobel Medical    Institute"
  .. .. .. .. ..$ city   : chr "Stockholm"
  .. .. .. .. ..$ country: chr "Sweden"
 $ :List of 13
  ..$ id             : chr "604"
  ..$ firstname      : chr "Erik Axel"
  ..$ surname        : chr "Karlfeldt"
  ..$ born           : chr "1864-07-20"
  ..$ died           : chr "1931-04-08"
  ..$ bornCountry    : chr "Sweden"
  ..$ bornCountryCode: chr "SE"
  ..$ bornCity       : chr "Karlbo"
  ..$ diedCountry    : chr "Sweden"
  ..$ diedCountryCode: chr "SE"
  ..$ diedCity       : chr "Stockholm"
  ..$ gender         : chr "male"
  ..$ prizes         :List of 1
  .. ..$ :List of 5
  .. .. ..$ year        : chr "1931"
  .. .. ..$ category    : chr "literature"
  .. .. ..$ share       : chr "1"
  .. .. ..$ motivation  : chr "\"The poetry of Erik Axel   Karlfeldt\""
  .. .. ..$ affiliations:List of 1
  .. .. .. ..$ : list()                             **<--problem here**

If you add some random data to that list, the code works fine.

nobels[[2]]$prizes[[1]]$affiliations[[1]]<-list(name="random data")

Use plyr package:

library (plyr)
mydf <- ldply(nobels, data.frame)

Upvotes: 1

Related Questions