CODEWITHSUNDEEP
d3.jscrossfilter
dmartinez
dmartinez

Reputation: 287

d3 - aggregate data giving a "step" value

I am trying to think about semantic zooming and data aggregation to some rendering/panning SVG's issues.

So whenever my view range (x domain) gets too big and its data get's too many elements I draw some "merged/aggregated"new data that represents my origianal one.

So, one way of reducing my data is aggregating it according to an "step" value.

Let's say I have an x.domain([1,1000]) and a filtered data with 1000 elements (from position 1 to 1000). I'll set this 1000 position/elements as my fixed amount of elements to be rendered.

IF the user zooms out to a new x.domain([1,2000)], instead of getting 2000 data elements, I'd like to aggregate (mean or median) somehow my data. In this case I'd have an step of 2 (i.e. domain size / fixed # of elements OR 2000/1000=2)

It seems to be a reasonable thing to do once too many rendered elements is slow and not a good thing to do using SVG. It can get big easily.

So my idea would be (in this scenario): aggregate values by step.

Let's say I have this data object:

{position:1, value:8}, {position:2, value:10}, {position:3, value:10}, {position:4, value:12}, {position:5, value:14}, {position:6, value:16}, ...

the resulting agreggated object would be:

{position:1+2/step, value:8+10/step}, {position:3+4/step,value:10+12/step}, {position:5+6/step, value:14+16/step}, ...

This way I'll always keep 1000 elements to be rendered. And once my samplying rate=1, it will still gives a good idea of the data.

My question is:

what would be the best solution to aggregate my data this way ? (once it is not aggreagated by an specific field (i.e. month, country, etc)).

Will d3.nest() be the right choise ? what about crossfilter ?

Any thoughts and/or help is highly appreciated.

Cheers., D

Upvotes: 0

Views: 463

Answers (1)

ahmohamed
ahmohamed

Reputation: 2970

While crossfilter is definitely a good option, I prefer a lightweight library simplify.js, especially if you are dealing with line charts or your data contain extreme points that may get filtered by your step function. simplify.js reduces your data so that the chart looks visually similar to the one created using all data points.

Upvotes: 1

Related Questions