d3 adding and removing nodes with force

Question

I've put together the following jfiddle based on some code I've seen in a book - http://jsfiddle.net/hiwilson1/o3gwejbx/2. Broadly speaking I follow what's happening, but there's a few aspects I don't follow.

svg.on("mousemove", function () {

    var point = d3.mouse(this), 
        node = {x: point[0], y: point[1]}; 

    svg.append("circle")
        .data([node])
            .attr("r", 1e-6)
            .transition()
            .attr("r", 4.5)
            .transition()
            .delay(1000)
            .attr("r", 1e-6)
            .remove();
        force.nodes().push(node); 
        force.start(); 
});

Here we build our new data point and append a circle with attributes x and y of this data point. I transition the nodes radius in and then out and then remove() it. Here's the bit I don't follow - BEFORE removing it the data point is added to the force.nodes() array, not the circle itself, just the data point. I then start() the force.

1. Why are we removing the circle BEFORE pushing the data point into the force.nodes() array.
2. Why are we pushing just the data point into the force.nodes() array, does it not need a reference to the circle? Or is the circle somehow just the visual representation of the data point and manipulating the data point manipulates the circle?
3. Why are we starting the force() each time we move the mouse? I thought the force() was ticking over in the background and didn't need to be restarted after appending every single node?

UPDATE: I think what I ultimately am looking for clarity on is what the force() layout is actually doing under the hood.

Theory: You give the force layout an array of nodes. For each data element, the x and y is either provided or arbitrarily assigned. Once the force is started, the array is constantly recalculated to move those x and y components according to the additional force properties applied, such as gravity and charge. The force layout has nothing to do with the visualisation of the circles themselves - you have to keep drawing them / refreshing their x and y locations to reflect the positions of the array values that the force is manipulating.

Is any of that correct?

Answer 1

I guess it's just a compact way of doing it but, not a really good learning example... For one thing, the nodes data are never removed and secondly, the method is a bit imperative and not really data driven.

The nodes number in this demo is the length property of the nodes array

		var w = 900, h = 400, nodes = [],
						indx = 0, show = false,

		svg = d3.select("body").append("svg")
		.attr("width", w)
		.attr("height", h),

		force = d3.layout.force()
		.nodes(nodes)
		.size([w, h])
		.gravity(0)
		.charge(1)
		.friction(0.7),

		outputDiv = d3.select("body").insert("div", "svg").attr("id", "output");

		$("#toggleShow").click(function (e) {
			d3.selectAll(".dead").attr("opacity", (show = !show) ? 0.2 : 0)
			$(this).text((show ? "don't " : "") + "show dead nodes")
		});
		$("#clear").click(function (e) {
			nodes.length = 0;
			d3.selectAll("circle").remove();
		});


		force.on("tick", function (e) {
			outputDiv.text("alpha:\t" + d3.format(".3f")(force.alpha())
				+ "\tnodes:\t" + force.nodes().length)
			var circles = svg.selectAll("circle").data(nodes, function (d) { return d.id })

			//ENTER
			//  direct
			//    data is there but the circle has been deleted by completion of transition
			//    replace the previously live node with a dead one
			//  idiomatic
			//    always zero size
			circles.enter().append("circle")
				.attr("r", 4.5)
				.attr("class", "dead")
			.attr("opacity", show ? 0.2 : 0);
			//UPDATE+ENTER
			circles
				.attr("cx", function (d) { return d.x; })
				.attr("cy", function (d) { return d.y; });
		});

		svg.on("mousemove", onMove)
		.on("touchmove", onMove)
		.on("touchstart", onMove);

		function onMove() {
		  d3.event.preventDefault();
		  d3.event.stopPropagation();
		  updateMethod.call(this)
		}

		function direct() {
			return function () {
			  var pointM = d3.mouse(this), pointT = d3.touches(this),
						point = pointT.length ? pointT[0] : pointM,
						node = { x: point[0], y: point[1], id: indx++ };

				svg.append("circle")
							.data([node])
									.attr("class", "alive")
									.attr("r", 1e-6)
									.transition()
									.attr("r", 4.5)
									.transition()
									.delay(1000)
									.attr("r", 1e-6)
									.remove();
				force.nodes().push(node);
				force.start();
			}
		} /*direct*/


		updateMethod = direct();

	body, html {
			width:100%;
			height:100%;
	}
		#vizcontainer {
			width: 100%;
			height: 100%;
		}

	 svg {
			outline: 1px solid red;
			width: 100%;
			height: 100%;
		}

		#output {
			pointer-events: none;  
			display: inline-block;
			z-index: 1;
			margin: 10px;
		}

		button {
			display: inline-block;
			margin: 10px;
		}
		.dead {
			fill: white;
			stroke: black;
			stroke-width: 1px;
		}

<button id="toggleShow" name="">show dead nodes</button>
	<button id="clear" name="clear">clear</button>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.4.11/d3.min.js"></script>

Even though the nodes have been removed by the transitions, they are still there in the nodes array and therefore, still acting in the force calculation. You can see that as a sort of a black hole effect as the count of (data) nodes builds up: a sink starts to develop due to the growing clump of invisible nodes.

In answer to your questions...

1. Why not? It works either way...
2. As you mention in your update, the layout is only providing the positions of the objects and has no reference to the objects themselves. Yes, that's up to you to manage.
3. If you use the browser developer tools to look at the elements of the array returned by force.nodes(), you will see that there is a lot of state added over and above the original x and y members, there is also state closured in the d3.force object such as distances, strengths and charges. All this has to be set up somewhere and not surprisingly, it's done in force.start(). So that's why you have to call force.start() every time you change the structure of the data. It's really not hard to track this stuff down if you RTFC, that's how you find out what's under the hood.

in terms of patterns, this would be more idiomatic for d3...

    ;(function() {
      var w = 900, h = 400, nodes = [], touch,

          svg = d3.select("#vizcontainer").append("svg")
          .attr("width", w)
          .attr("height", h),

          force = d3.layout.force()
          .size([w, h])
          .gravity(0)
          .charge(1)
          .friction(0.7),

          outputDiv = d3.select("body").insert("div", "#vizcontainer").attr("id", "output").attr("class", "output"),
          touchesDiv = d3.select("body").insert("div", "#output").attr("id", "touches")
          .style("margin-right", "10px").attr("class", "output");


      force.on("tick", function (e) {

        outputDiv.text("alpha:\t" + d3.format(".3f")(force.alpha())
          + "\tnodes:\t" + force.nodes().length)

        svg.selectAll("circle")
        .attr("cx", function (d) { return d.x; })
        .attr("cy", function (d) { return d.y; });
      });

      svg.on("mousemove", onMove);
      svg.on("touchmove", onTouch);
      svg.on("touchstart", onTouch);

      function onMove() {
        updateMethod.call(this)
      }
      function onTouch() {
        d3.event.preventDefault();
        d3.event.stopPropagation();
        updateMethod.call(this)
      }

      function idiomatic() {
        force.nodes(nodes);
        return function () {
          var pointM = d3.mouse(this), pointT = d3.touches(this),
              point = pointT.length ? pointT[0] : pointM,
          node = { x: point[0], y: point[1] };

          //touchesDiv.text(pointT.length ? pointT : "mouse");

          nodes.push(node);

          svg.selectAll("circle")
          .data(nodes)
          .enter().append("circle")
          .attr("r", 1e-6)
          .transition("in")
          .attr("r", 4.5)
          .transition("out")
          .delay(1000)
          .attr("r", 1e-6)
          .remove()
          .each("end.out", (function (n) {
            return function (d, i) {
              //console.log("length: " + nodes.length + "\tdeleting " + i)
              var i = nodes.indexOf(n);
              nodes.splice(i, 1)
            }
          })(node));

          force.start();
        }
      } /*idiomatic*/


      updateMethod = idiomatic();
    })()

  body, html {
      width:100%;
      height:100%;
  }
    #vizcontainer {
      width: 100%;
      height: 100%;
    }

   svg {
      outline: 1px solid red;
      width: 100%;
      height: 100%;
    }

    .output {
      pointer-events: none;  
      display: inline-block;
      z-index: 1;
      margin: 10px;
    }

<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.4.11/d3.min.js"></script>
  <div id="vizcontainer"></div>

This is a subset of the often mentioned general update pattern. In this case however, only the enter selection is considered, because the data is driving this phase only. Exit behaviour is pre-programmed into the transitions, so this is a special case and the data clean-up needs to be driven by the timing of the transitions. Using the end event is one way to do that. It's important to note that each node has its own individual transition, so that it works out nicely in this case.

And yes, your theory is correct.

Answer 2

First: Remove() is called when transition are finished (including delay). In the example, circles grow and become small again. At that point, it is removed with .remove().

Second: This piece of code is what selects all circles again each tick of the force and moves the circles:

force.on("tick", function () {
    svg.selectAll("circle")
    .attr("cx", function (d) {return d.x;})
    .attr("cy", function (d) {return d.y;});
});

Third: A new circle is created on each mouse move, and added to the existing force.