CODEWITHSUNDEEP
javascriptcsvd3.jsnestedhierarchical-data
alexc
alexc

Reputation: 1310

How to organise/nest data for d3.js chart output

I'm looking for some advice on how to effectively use large amounts of data with d3.js. Lets say for instance, I have this data set taken from a raw .csv file (converted from excel);

EA
,Jan_2016,Feb_2016,Mar_2016
Netherlands,11.7999,15.0526,13.2411
Belgium,25.7713,24.1374
France,27.6033,23.6186,20.2142

EB
,Jan_2016,Feb_2016,Mar_2016
Netherlands,1.9024,2.9456,4.0728
Belgium,-,6.5699,7.8894
France,5.3284,4.8213,1.471

EC
,Jan_2016,Feb_2016,Mar_2016
Netherlands,3.1499,3.1139,3.3284
Belgium,3.0781,4.8349,5.1596
France,16.3458,12.6975,11.6196

Using csv I guess the best way to represent this data would be something like;

Org,Country,Month,Score
EA,Netherlands,Jan,11.7999
EA,Belgium,Jan,27.6033
EA,France,Jan,20.2142
EA,Netherlands,Feb,15.0526
EA,Belgium,Feb,25.9374
EA,France,Feb,23.6186
EA,Netherlands,Mar,13.2411
EA,Belgium,Mar,24.1374
EA,France,Mar,20.2142

This seems very long winded to me, and would use up a lot of time. I was wondering if there was an easier way to do this?

From what I can think of, I assume that JSON may be the more logical choice?

And for context of what kind of chart this data would go into, I would be looking to create a pie chart which can update the data depending on the country/month selected and comparing the three organisations scores each time.

(plnk to visualise) http://plnkr.co/edit/P3loEGu4jMRpsvTOgCMM?p=preview

Thanks for any advice, I'm a bit lost here.

Upvotes: 2

Views: 678

Answers (1)

tarulen
tarulen

Reputation: 2100

I would say the intermediary step you propose is a good one for keeping everything organized in memory. You don't have to go through a csv file though, you can just load your original csv file and turn it into an array of objects. Here is a parser:

d3.text("data.csv", function(error, dataTxt) { //import data file as text first
 var dataCsv=d3.csv.parseRows(dataTxt); //parseRows gives a 2D array
 var group=""; // the current group header ("organization")
 var times=[]; //the current month headers
 var data=[];  //the final data object, will be filled up progressively
 for (var i=0;i<dataCsv.length;i++) {
    if (dataCsv[i].length==1 ) { //group name
       if ( dataCsv[i][0] == "") 
          i++; //remove empty line
       group = dataCsv[i][0]; //get group name
       i++;
       times = dataCsv[i];//get list of time headings for this group 
       times.shift(); // (shift out first empty element)
     } else {
       country=dataCsv[i].shift(); //regular row: get country name
       dataCsv[i].forEach(function(x,j){ //enumerate values
         data.push({ //create new data item
           Org: group,  
           Country: country,
           Month: times[j],
           Score: x
         })
       }) 
    }
 }

This gives the following data array:

data= [{"Org":"EA","Country":"Netherlands","Month":"Jan_2016","Score":"11.7999"},
       {"Org":"EA","Country":"Netherlands","Month":"Feb_2016","Score":"15.0526"}, ...]

This is IMO the most versatile structure you can have. Not the best for memory usage though.

A simple way to nest this is the following:

d3.nest()
  .key(function(d) { return d.Month+"-"+d.Country; })
  .map(data);

It will give a map with key-values such as:

"Jan_2016-Netherlands":[{"Org":"EA","Country":"Netherlands","Month":"Jan_2016","Score":"11.7999"},{"Org":"EB","Country":"Netherlands","Month":"Jan_2016","Score":"1.9024"},{"Org":"EC","Country":"Netherlands","Month":"Jan_2016","Score":"3.1499"}]

Use entries instead of mapto have an array instead of a map, and use a rollup function if you want to simplify the data by keeping only the array of scores. At this point it is rather straightforward to plug it into any d3 drawing tool.

PS: a Plunker with the running code of this script. Everything is shown in the console.

Upvotes: 2

Related Questions