Why is my server failing to parse this JSON code?

Question

I had some JSON code (since lost) that my server parsed successfully. I kept the structure and just added/altered some data entries. Now my code is returning the following error when I try to parse it:

JSON::ParserError in Static#testing4
Showing /home/user/website/app/views/static/testing4.html.erb where line #44 raised:

399: unexpected token at '  {
        "title": "TITLE1",
        "capital": "Rome",
        "length": 2.7,
        "date": "2017-1-2",
        "tags": [
          "Culture"
        ]
    }],
    "2": [
      {
        "title": "TITLE2",
        "capital": "Paris",
        "age": 1.2,
        "date": "2017-2-1",
        "tags": [
          "Culture"
        ]
    }]
}
'

Here's my code. The problem is that the {"1":[ at the beginning isn't being parsed for some reason. There is no associated controller action for this view.


<%
require 'json'
require 'pp'

json = <
<%= obj = JSON.parse(json) %>

Trying to output it without parsing it has the following result:

<%
require 'json'
require 'pp'

json = <
PUTS: <%= puts json %>

PP: <%= pp json %>


---output---

PUTS: 
PP: { "1": [   {     "title": "TITLE1",     "capital": "Rome",     "length": 2.7,     "date": "2017-1-2",     "tags": [       "Culture"     ] }], "2": [   {     "title": "TITLE2",     "capital": "Paris",     "age": 1.2,     "date": "2017-2-1",     "tags": [       "Culture"     ] }] }

7stud · Accepted Answer

Is jsonlint unreliable, or is something else going on?

jsonlint is reliable, and ruby has no trouble parsing the json you posted:

require 'json'
require 'pp' 

json = <
  [{"title"=>"TITLE1",
    "capital"=>"Rome",
    "length"=>2.7,
    "date"=>"2017-1-2",
    "tags"=>["Culture"]}],
 "2"=>
  [{"title"=>"TITLE2",
    "capital"=>"Paris",
    "age"=>1.2,
    "date"=>"2017-2-1",
    "tags"=>["Culture"]}]}

However, non-printing characters won't necessarily be visible if you look at a file or output a string:

test = "\001hello"
puts test
p test
    
--output:--
hello
"\u0001hello"

You'll get a parse error here:

require 'json'
require 'pp' 

json = <


Even though the json looks fine when you output it:
require 'json'
require 'pp' 

json = <'

So you may want to read in your json from the file, then use p to output the text, then examine the text for any strange characters.
Also, please copy and paste your full error message--don't ever type out what you think an error message says, then claim that is what your program produced. I can sort of duplicate the error message you posted with this json:
json = <'

Response to comment 1:
The error message contains the json that rails is seeing, and here it is:
399: unexpected token at '  {   #<===Opening brace, which starts a javascript object.
        "title": "TITLE1",
        "capital": "Rome",
        "length": 2.7,
        "date": "2017-1-2",
        "tags": [
          "Culture"
        ]
    }],   #<====First, a closing brace that ends the javascript object, followed by a closing bracket, ending what?
    "2": [
      {
        "title": "TITLE2",
        "capital": "Paris",
        "age": 1.2,
        "date": "2017-2-1",
        "tags": [
          "Culture"
        ] 
    }]
}

If I put that invalid json in a rails view:
<%
json = <

...then enter the route in my browser that will display that view, here is the error message I get in my browser:
Showing /Users/7stud/rails_projects/test4/app/views/users/index.html.erb where line #27 raised:

784: unexpected token at '{
        "title": "TITLE1",
        "capital": "Rome",
        "length": 2.7,
        "date": "2017-1-2",
        "tags": [
          "Culture"
        ]
    }],
    "2": [
      {
        "title": "TITLE2",
        "capital": "Paris",
        "age": 1.2,
        "date": "2017-2-1",
        "tags": [
          "Culture"
        ]
    }]
}
'

Look familiar?  If instead you had an error elsewhere in your json, like here:
<%
json = <

Then you would see an error message like the following in your browser:
Showing /Users/7stud/rails_projects/test4/app/views/users/index.html.erb where line #32 raised:

784: unexpected token at '{    
  "1": [   #<===Note the presence of "1": [, unlike in your error message.
      {
        "title": "TITLE1",
        "capital": "Rome",
        "length": 2.7,
        "date": "2017-1-2",
        "tags": [
          "Culture"
        ]
      }
  ],
  "2":    
    {
        "title": "TITLE2",
        "capital": "Paris",
        "age": 1.2,
        "date": "2017-2-1",
        "tags": [
          "Culture"
        ]
    }
  ]
}
'

Therefore, the "1": [ is not present in the json that rails sees.  If you need more help, you need to copy and paste the code for:

Your controller and the action.
The view for the action.

Response to comment 2:
A rails view is able to render the following json without error (all whitespace removed):

<%
require 'json'
require 'pp'

json = <
<%= obj = JSON.parse(json) %>

And, a rails view is able to render the following json without error (all whitespace added back in):

<%
require 'json'
require 'pp'

json = <
<%= obj = JSON.parse(json) %>

However, for some reason a rails view cannot render your json:

<%
require 'json'
require 'pp'

json = <
<%= obj = JSON.parse(json) %>

After some further investigation, your json has a mixture of ascii spaces (20 in hex notation) and a type of UTF-8 space character called an EM SPACE (U+2003, e2 80 83 in UTF-8).  The first EM SPACE in your json occurs here:
{
    "1": [
      {
    ^
    |
 Right there

...which coincidentally is where the rails error message is cutting off your json.  Immediately preceding that opening brace (7b in hex notation) is an ascii space(20 in hex notation), and the character preceding the ascii space is a three byte UTF-8 character called an EM SPACE(U+2003, e2 80 83 in UTF-8).  Characters encoded in the UTF-8 encoding are supposed to be legal json.  jsonlint accepts the EM SPACE, but rails chokes on it.
As a short example of the problem, the following javascript array has an EM SPACE after each comma:
  
<%
json = "[1, 2, 3]"
%>

<%= obj = JSON.parse(json) %>

Here's the error in my browser:
419: unexpected token at ' 2, 3]'

The rails error chops off the json up to the first EM SPACE.
It turns out that while UTF-8 characters are allowed in json, there are only a few ascii white space characters that are allowed between json terms--not just any UTF-8 character.  See RFC 7159.  Using an EM SPACE between json terms is conceptually equivalent to using, say, a z between terms.  As a result, the json containing the EM SPACE characters is illegal json.  It would be legal to use an EM SPACE inside a string, but not between json terms.
The following jsonlinter reports that the json with the EM SPACE characters is invalid:
http://www.freeformatter.com/json-validator.html
It appears that some jsonlinters are more reliable than others.
A solution:
You can use gsub() to replace all the EM SPACE characters with ascii spaces:
...
...
END_OF_JSON

clean_json = json.gsub("\u2003", ' ')

%>
<%= obj = JSON.parse(clean_json) %>

If you got that json from a service somewhere, you should shoot them an email and let them know that the json they are sending out is illegal.

Why is my server failing to parse this JSON code?

Answers (1)

Related Questions