CODEWITHSUNDEEP
javascriptparsingd3.jstext-parsing
jackdbd
jackdbd

Reputation: 5061

Parse resource in d3.queue

I am using d3.queue to load two different resources before drawing a visualization.

If the resources do not need to be parsed, the task can be achieved pretty easily:

d3.queue()
  .defer(d3.text, 'file1.txt')
  .defer(d3.json, 'file2.json')
  .await(draw);

Now, suppose that file.txt has the following structure:

STATE|STUSAB|STATE_NAME|STATENS
01|AL|Alabama|01779775
02|AK|Alaska|01785533
04|AZ|Arizona|01779777

I know I can use d3.dsvFormat('|') to parse this file, but how can I do it in a queue?

Here is a JSFiddle

The file I am using can be found here: National FIPS and GNIS Codes File

Upvotes: 1

Views: 366

Answers (2)

jackdbd
jackdbd

Reputation: 5061

I had a look at the d3-request code and I implemented a function that loads and parses a pipe-separated .txt file. It's basically the same code of json.js.

import * as d3RequestType from '../../node_modules/d3-request/src/type';

const d3Psv = d3RequestType.default('text/plain', (xhr) => {
  const psv = d3.dsvFormat('|');
    return psv.parse(xhr.responseText);
});

const queue = d3.queue();
queue
  .defer(d3Psv, 'file1.txt')
  .defer(d3.json, 'file2.json')
  .await(draw);

In another project I had to load a file where the values were separated with semicolons and I used the same approach.

Note that you can also pass a rowFunction as the second argument of psv.parse.

Upvotes: 0

altocumulus
altocumulus

Reputation: 21578

You cannot do that parsing in a deferred tasked like you do e.g. for d3.json as there is no convenience method for requesting and parsing your content at once. When dealing with delimiter separated values D3 offers convenience methods for only the comma (d3.csv) and tabs (d3.tsv). These will request the resource and parse the contents in a single run. The await callback will then be called with the already parsed contents.

Since you are using a different delimiter, namely a pipe "|" character, you will have to do these two steps one after the other. In a queue you can still use d3.text to request the textual contents of your file. The parsing, however, will have to be done in the await callback by using d3.dsvFormat("|").parse. The necessary pieces are already contained in your code; you just have to use psv to do the parsing in your draw() function:

const psv = d3.dsvFormat('|');

const draw = (error, data1, data2) => {
  console.log(psv.parse(data1));         // Do the parsing in await
  console.log(data2);
}

const queue = d3.queue();
queue
  .defer(d3.text, url1)                  // Load textual contents
  .defer(d3.json, url2)
  .await(draw);

Have a look at the updated JSFiddle for a working example.

Upvotes: 1

Related Questions