CODEWITHSUNDEEP
jsondebuggingjq
rubo77
rubo77

Reputation: 20817

Debug a large json file that is in one line

I have a 2MB json file that is only all in one line and now I get an error using jq:

$ jq .<nodes.json

parse error: Invalid literal at line 1, column 377140

How do I debug this on the console? To look at the mentioned column, I tried this:

head -c 377139 nodes.json|tail -c 1000

But I cannot find any error with a wrong t there, so it seems it is not the correct way to reach the position in the file.

How can I debug such a one-liner?

Upvotes: 1

Views: 1578

Answers (2)

knb
knb

Reputation: 9295

I see you are on a shell prompt. So you could try perl, because your operating system has it pre-installed, presumably.

cat nodes.json | json_xs -f json -t json-pretty

This tells the json_xs command line program to parse the file and prettify it.

If you don't have json_xs installed, you could try json_pp (pp is for pure-perl).

If you have neither, you must install the JSON::XS perl module with this command:

sudo cpanm JSON::XS
[sudo] password for knb: 
--> Working on JSON::XS
Fetching http://www.cpan.org/authors/id/M/ML/MLEHMANN/JSON-XS-3.01.tar.gz ... OK
Configuring JSON-XS-3.01 ... OK
Building and testing JSON-XS-3.01 ... OK
Successfully installed JSON-XS-3.01 (upgraded from 2.34)
1 distribution installed

This installs JSON::XS and a few helper scripts, among them json_xs and json_pp.

Then you can run this simple one-liner:

cat dat.json | json_xs -f json -t json-pretty

After misplacing a parenthesis to force a nesting-error somewhere in the valid json file dat.json I got this:

cat dat.json | json_xs -f json -t json-pretty
'"' expected, at character offset 1331 (before "{"A_DESC":"density i...") at /usr/local/bin/json_xs line 181, <STDIN> line 1.

Maybe this is more informative than the jq output.

Upvotes: 0

rubo77
rubo77

Reputation: 20817

cut the file into more lines with

cat nodes.json|cut -f 1- -d} --output-delimiter=$'}\n'>/tmp/a.json

and analyse /tmp/a.json with, then you get an error with line nr:

parse error: Invalid literal at line 5995, column 47

use less -N /tmp/a.json to find that line

Upvotes: 2

Related Questions