d3.js: how to join data from more sources

Question

gs.csv (data1):

id, name, x_value, y_value
1, fruits, 60, 60
2, vegetables, 70, 70
...

circles.csv (data2):

id, name, value, cx_value, cy_value
1, fruits, apple, 10, 10
2, fruits, pear, 20, 20
3, fruits, strawberry, 30, 30
4, vegetables, carrot, 40, 40
5, vegetables, celery, 50, 50
...

I have two data files as above. gs.csv (data1) contains only data about g elements and circles.csv (data2) contains only data about circles and I need a proper way to join them and create following:

    <g class="groups" id="fruits" transform="translate(60,90)">
      <circle class="some" id="apple" cx="10" cy="10"/>
      <circle class="some" id="pear" cx="20" cy="20"/>
      <circle class="some" id="strawberry" cx="30" cy="30"/>
      ...
    </g>
    <g class="groups" id="vegetables" transform="translate(70,70)">
      <circle class="some" id="carrot" cx="40" cy="40">
      <circle class="some" id="celery" cx="50" cy="50">
      ...
    </g>

I can't join the files because they are more complicated. I think that code should be something similar to:

    d3.csv("gs.csv", function(data1) {
     d3.csv("circles.csv", function(data2) {
       var svg = ...
       var groups = svg.selectAll(".groups")
                  .data(data1)
                  .enter().append("g")
                  .attr("class", "groups")
                  .attr("id", function(d) { return d.name; })
                  .attr("transform", function(d){return "translate(" + d.x_value + "," + d.y_value + ")"});

        groups.selectAll(".some")
        .data(data2, function(d) { return d.id; })
        .enter().append("circle")
        .attr("class", "some")
        .attr("id", function(d) { return d.value; })
        .attr("cx", function(d) { return d.cx_value; })
        .attr("cy", function(d) { return d.cy_value; });
    })
  });

Two solutions come into my mind but always there is some catch: First, I tried to filter lines from data2 in the second .data() but I don't know how to access attributes of the actual group to filter all circles except those with the same name value. Second, I tried to d3.nest data2 with key d.name and enter these into groups, but it overwrite original __ data_. So, I tried to add key and values to _ data__ of each group but without success.

Answer 1

Another way to do this is using nest. I think it will work for your problem if you have exactly the same names (fruits, vegetables, etc.) in both data files. It has the advantage of being linear time instead of quadratic, if that matters. (If the two data files have the same names but in a different order, you will have to sort them first, making it O(nlogn).)

var nest = d3.nest()
             .key(function(d) {return d.name;})
             .entries(data2);

This separates out the fruit rows and vegetable rows into different arrays.

var zip = d3.zip(data1,nest);

This combines the two files into one data set. It forms an array with one item per unique name. (This is the part that requires the names to be the same, in the same order.) Each item of the array is an array with two items: The first is the row from data1, and the second is a group of rows from data2 (in a nest structure). Now you can join the data to your SVG:

        svg.selectAll("g")
           .data(zip)
           .enter()
           .append("g")
           .attr("class","groups")
           .attr("id",function(d){return d[0].name;})
           .attr("transform",function(d){return "translate("+d[0].x_value+","+d[0].y_value+")"})
           .selectAll("circle")
           .data(function(d){return d[1].values;})
           .enter()
           .append("circle")
           .attr("class", "some")
           .attr("id", function(d) { return d.value; })
           .attr("cx", function(d) { return d.cx_value; })
           .attr("cy", function(d) { return d.cy_value; });

d[0] is a row of your data1 file, and d[1].values is a group of rows of your data2 file.

No need for .each, .filter, or a helper function.

I learned how to do it from these pages:

• https://github.com/mbostock/d3/wiki/Arrays#nest
• https://bost.ocks.org/mike/nest/

Answer 2

The simplest way to do this is to put your data groups in a list and then operate on them, as follows:

groupData = [data1, data2];

groups = svg.selectAll('g')
    .data(groupData)
    .append('g')

Now you have two groups, each of which has data appended for the things you want inside the group. You can make a function to append your circles based on the data in a group, and then call it for each of your groups.

function makeCircles(d){
    d3.select(this).selectAll('circle')
        .data(d)
        .append('circle')
        .attr('rx',function(D){return D.circleRadius})

groups.each(makeCircles);

This will give you the relevant circles for each group. Note that, within the selectAll that we've created within makeCircles, D refers to the data associated with a given circle. We could use d instead, but it's best to avoid variable confusion between the group data d and the circle data D. This way we can also use both in any of the functions that define attributes of the circles.

If we want the first data to contain the second data, we can use the same function, but with a filter based on d replacing D itself. The code would look like this:

var root = d3.select('body').append('svg'),

    data1 = [{'name':'foo','value':10},{'name':'foo','value':3},{'name':'foo','value':8},{'name':'bar','value':10},{'name':'bar','value':1},{'name':'bar','value':15}],

    data2 = [{'name':'foo','color':'green','x':10},{'name':'bar','color':'blue','x':70}];

console.log('foo')

var groups = root.selectAll('g')
    .data(data2)
    .enter()
    .append('g')
    .attr('transform',function(d){return 'translate(' + d.x + ',10)'})
    .each(addCircles);

function addCircles(d){
    d3.select(this).selectAll('circle')
        .data(data1.filter(function(D){return D.name == d.name }))
        .enter()
        .append('circle')
        .attr('r',5)
        .attr('cx', 0)
        .attr('cy', function(D){return D.value * 30})
        .style('fill',d.color)
}

I've made a fiddle here.

Answer 3

You can utilize the queue library for d3 by including

<script src="http://d3js.org/queue.v1.min.js"></script>

in your HTML file. Then within your javascript include

queue()
   .defer(d3.csv, "gs.csv")
   .defer(d3.csv, "circles.csv")
   .await(ready);

function ready(error, gs, circles){
...
}

Now, within that ready function, you can access both data sets. You could first create the groups:

var groups = d3.selectAll(".groups")
    .data(gs)
  .enter().append("g")
    .attr("class", "groups")
    .attr("id", function(d) { return d.name; })
    .attr("transform", function(d){return "translate(" + d.x_value + "," + d.y_value + ")"});

And then

groups.selectAll(".some")
    .data(circles)
  .enter().append("circle")
    .attr("class", "some")
    .attr("id", function(d) { return d.value; })
    .attr("cx", function(d) { return d.cx_value; })
    .attr("cy", function(d) { return d.cy_value; });

I think though that this isn't quite working for creating only the relevant circles in each group. Some more thought has to be given for how exactly to accomplish this.