CODEWITHSUNDEEP
ruby-on-railsjsonencodingunicoderuby-on-rails-4
jacob
jacob

Reputation: 365

Why does to_json escape unicode automatically in Rails 4?

Rails 3:

{"a" => "<br/>"}.to_json
=> "{\"a\":\"<br/>\"}"

Rails 4:

{"a" => "<br/>"}.to_json
=> "{\"a\":\"\\u003Cbr/\\u003E\"}"

WHY???

It appears to be causing the error

Encoding::UndefinedConversionError: "\xC3" from ASCII-8BIT to UTF-8

When my Rails 3 app tries to parse JSON generated by my rails 4 app.

Upvotes: 22

Views: 8573

Answers (4)

guero64
guero64

Reputation: 1049

Was having a similar problem with Rails 7 sending "<" in JSON output like:

..., "legend":[{"text":"<96.8%","color":"#FFAFFF"},{"text":"96.8% to 98.8%","color":"#E37DE3"},{"text":"98.8% to 100%","color":"#BA50BA"}], ...

from something like:

{entry: dataset.entry, legend: dataset.legend, ...

The "<" sign was showing up "legend":[{"text":"\u003c96.8%", ...


In my case `JSON.generate({entry: ...})` fixed the issue

Upvotes: 1

Dom Eden
Dom Eden

Reputation: 131

I encountered this issue too and as others have mentioned, it's caused by using the ActiveSupport to_json method. To resolve, use the JSON gem directly with JSON.generate(data) where data is an Array or Hash. See https://github.com/flori/json for all JSON gem documentation.

Upvotes: 4

mahemoff
mahemoff

Reputation: 46389

You can retain the original string with JSON::dump:

JSON::dump "a" => "<br/>"
=> "{\"a\":\"<br/>\"}"

JSON::dump "a" => "x&y"
=> {\"a\":\"x&y\"}" # instead of x\u0026y

Use it with care for the reasons bobince mentions and particularly avoid it with any user-generated input (or at least make sure that's sanitized).

Here's an example I encountered where it's a legitimate use. Generating a JavaScript hash argument in a helper function:

# application_helper.rb

def widget_js(post)
  options = {
    color: ColorCalculator(post.color).to_rgb_hex,
    ...
  }
  "third_party_widget(#{JSON::dump options});"
end

Upvotes: 12

bobince
bobince

Reputation: 536379

WHY???

To defend against a common weakness in web applications. If you say in an HTML page eg:

<script type="text/javascript">
    var something = <%= @something.to_json.html_safe %>;
</script>

then you might think you're fine because you've JSON-escaped the data you're injecting into JavaScript. But actually you're not safe: aside from JSON syntax you also have surrounding HTML syntax, and in an HTML script block </ is in-band signalling. Practically, if @something contains the string </script> you've got a cross-site scripting vulnerability as this comes out:

<script type="text/javascript">
    var something = {"attack": "abc</script><script>alert('XSS');//"};
</script>

The first script block ends halfway through the string (leaving an unclosed string literal syntax error) and the second <script> is treated as a new script block and the potentially-user-submitted content within it executed.

Escaping the < character to \u003C is not required by JSON but it is a perfectly valid alternative and it automatically avoids this class of problems. If a JSON parser rejects it, that is a severe bug in the reader.

What is the code that is producing that error? I'm not convinced the error is anything to do with the <-escaping, as it is talking about byte 0xC3 rather than 0x3C. That could be indicative of a string with UTF-8 encoded content not having been marked as UTF-8... maybe you need a force_encoding("UTF-8") on the input?

Upvotes: 17

Related Questions