Write a JSON file for Array of Arrays

Question

I have several nx3 arrays to be written in JSON, so that they can be further read in Ruby.

I completed code to this point, but JSON.parse is returning an error when the second array is written into the file. How do I write the JSON such that each array can be read seperately?

Here is the code and the written JSON file.

require 'json'
wp = []
wp[0]=[704/4,1124/4,0]
wp[1]=[704/4,1608/4,0]
wp[2]=[942/4,1608/4,0]



a = wp.map{|wp|{x: wp[0].to_i, y:wp[1].to_i, z:wp[2].to_i}}

wp2 = []
wp2[0]=[7055/4,1124/4,0]
wp2[1]=[704/4,1608/4,0]
wp2[2]=[942/4,1608/4,0]
wp2[3]=[942/4,2107/4,0]

a2 = wp2.map{|wp2|{x: wp2[0].to_i, y:wp2[1].to_i, z:wp2[2].to_i}}



File.open(".../temp.json","w") do |f|
  f.write(a.to_json)
end


File.open(".../temp.json","a+") do |f|
  f.write(a2.to_json)
end

f = open(".../temp.json")
jon  = f.read
#This completes teh writing

psd = JSON.parse(jon)


psd.each do |xo|
  print xo["y"].to_i,"\n"
end

When only a single array is written in the file it is read correctly, but as the second array is written it writes correctly, but gives an error:

(JSON::ParserError)
    from /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/json/common.rb:155:in `parse'
    from jsont.rb:35:in `<main>'
"The JSON.parse(jon) gives Error."


[{"x":176,"y":281,"z":0},{"x":176,"y":402,"z":0},{"x":235,"y":402,"z":0},{"x":235,"y":526,"z":0},{"x":322,"y":526,"z":0},{"x":448,"y":526,"z":0},{"x":508,"y":526,"z":0},{"x":508,"y":640,"z":0},{"x":577,"y":640,"z":0},{"x":577,"y":669,"z":0},{"x":718,"y":669,"z":0}][{"x":1763,"y":281,"z":0},{"x":176,"y":402,"z":0},{"x":235,"y":402,"z":0},{"x":235,"y":526,"z":0},{"x":322,"y":526,"z":0},{"x":448,"y":526,"z":0},{"x":508,"y":526,"z":0},{"x":508,"y":640,"z":0},{"x":577,"y":640,"z":0},{"x":577,"y":669,"z":0},{"x":718,"y":669,"z":0}]

You'll notice that in the JSON file, two arrays have been concatenated. I know this is causing the error. But how can I correct it, such that I may be able to access each coordinate in the form of a two dimensional array? I don't know the number of arrays to be written before.

Answer 1

Maybe this will help you learn how to fish...

You can't write JSON output as two separate simple writes, and expect the parser to be able to interpret the data correctly. JSON, whether prettified and written as multiple lines, or written as a stream as a single line, is still an object, either an array or a hash. Both have to have their wrapping [...] or {...} and a continuation character showing there are more elements ',', both of which are what you're missing as you're writing the data.

Consider this bit of code and its output:

require 'json'

a_of_h_1 = [{'a' => 1}, {'b' => 2}]
a_of_h_2 = [{'c' => 3}, {'d' => 4}]

File.open('foo', 'w') do |fo|
  fo.puts JSON[a_of_h_1]
  fo.puts JSON[a_of_h_2]
end

Looking at the file in the shell it looks like this:

[{"a":1},{"b":2}]
[{"c":3},{"d":4}]

If I try to read that file:

JSON[File.read('foo')]

I get:

unexpected token at '[{"c":3},{"d":4}] (JSON::ParserError)

That's because there is not a containing/wrapping set of delimiters telling the parser that the data is part of a single object and there's not an intermediate ',' telling the parser that there is additional data. I could write the same data as an array of arrays of hashes:

File.open('foo', 'w') do |fo|
  fo.puts JSON[[a_of_h_1, a_of_h_2]]
end

Which results in a file that looks like:

[[{"a":1},{"b":2}],[{"c":3},{"d":4}]]

Note the containing [...] and the separating ','.

Now the parser is happy and is able to retrieve the object(s) correctly:

(a_a1, a_a2) = JSON[File.read('foo')]
a_a1 # => [{"a"=>1}, {"b"=>2}]
a_a2 # => [{"c"=>3}, {"d"=>4}]

Ok, back to your code. It's easier/safer/better to let the parser generate the stream for you. If you can't do that, then you have to be aware of how the parser needs to see the data, and then create the file in that format, which isn't that hard.

Again, starting out by looking to see what the generator does with a simple version of your data structure:

puts JSON[[a_of_h_1, a_of_h_2]]
# >> [[{"a":1},{"b":2}],[{"c":3},{"d":4}]]

it's easy to figure out how to build the file's format. I'd strongly recommend having the generator do as much of the heavy-lifting as possible, so I'm letting it serialize the individual sub-arrays:

a_of_a_of_h = []
a_of_a_of_h << a_of_h_1
a_of_a_of_h << a_of_h_2

File.open('foo', 'w') do |fo|
  fo.puts '['
  a_of_a_of_h[0..-2].each do |a_of_h|
    fo.puts a_of_h.to_json
    fo.puts ','
  end
  fo.puts a_of_a_of_h[-1].to_json
  fo.puts ']'
end

That code writes the opening '[', writes all the sub-array and adds the ',' character, then repeats up to the second-to-last element, then writes the last element and the closing ']'. Now the file looks like:

[
[{"a":1},{"b":2}]
,
[{"c":3},{"d":4}]
]

It's not pretty but it works:

JSON[File.read('foo')]
# => [[{"a"=>1}, {"b"=>2}], [{"c"=>3}, {"d"=>4}]]

If I clean up the code a bit I get:

File.open('foo', 'w') do |fo|
fo.puts '['
fo.puts a_of_a_of_h.map(&:to_json).join(',')
fo.puts ']'
end

map(&:to_json) applies the to_json method to each of the sub-arrays, then join(',') appropriately adds ',' between all the serialized elements. It's important to note that join doesn't add the comma after the last element, like a naive algorithm would.

Looking at the file:

[
[{"a":1},{"b":2}],[{"c":3},{"d":4}]
]

Reading and reparsing the data:

JSON[File.read('foo')]
# => [[{"a"=>1}, {"b"=>2}], [{"c"=>3}, {"d"=>4}]]

At this point it's easy to reassign the incoming data back to variables:

(ary_a, ary_b) = JSON[File.read('foo')]
# => [[{"a"=>1}, {"b"=>2}], [{"c"=>3}, {"d"=>4}]]
ary_a # => [{"a"=>1}, {"b"=>2}]
ary_b # => [{"c"=>3}, {"d"=>4}]

In my real code I'd write it something like:

File.open('foo', 'w') do |fo|
  fo.puts '[', fo.puts a_of_a_of_h.map(&:to_json).join(','), ']'
end

if I had to iterate over the arrays due to memory constraints or requirements. Otherwise I'd throw it at the JSON generator and let it chew through it all:

File.write('foo', JSON[a_of_a_of_h])
JSON[File.read('foo')]
# => [[{"a"=>1}, {"b"=>2}], [{"c"=>3}, {"d"=>4}]]

Answer 2

A proper array in JSON (even array of arrays) begins with a [, ends with a ], and between every two items there is a ,.

So, the following is not a valid JSON:

[1, 2, 3][4, 5, 6]

While this is:

[[1, 2, 3], [4, 5, 6]]

To solve your problem, you can write each array in a new line, and then when you read them back, you can read them line by line, parsing each separately:

File.open(".../temp.json","w") do |f|
  f.puts(a.to_json)
end


File.open(".../temp.json","a+") do |f|
  f.puts(a2.to_json)
end

This will create a file which looks something like this:

[{"x":176,"y":281,"z":0},{"x":176,"y":402,"z":0},{"x":235,"y":402,"z":0},{"x":235,"y":526,"z":0},{"x":322,"y":526,"z":0},{"x":448,"y":526,"z":0},{"x":508,"y":526,"z":0},{"x":508,"y":640,"z":0},{"x":577,"y":640,"z":0},{"x":577,"y":669,"z":0},{"x":718,"y":669,"z":0}]
[{"x":1763,"y":281,"z":0},{"x":176,"y":402,"z":0},{"x":235,"y":402,"z":0},{"x":235,"y":526,"z":0},{"x":322,"y":526,"z":0},{"x":448,"y":526,"z":0},{"x":508,"y":526,"z":0},{"x":508,"y":640,"z":0},{"x":577,"y":640,"z":0},{"x":577,"y":669,"z":0},{"x":718,"y":669,"z":0}]

Which you can later read like this:

arrs = File.readlines(".../temp.json").map do |line|
  JSON.parse(line)
end

This should result in an array of arrays.

Answer 3

You're writing a bunch of consecutive JSON strings to a file with no delimiter.

Your options are either:

1. create a single array of all your arrays, convert that to JSON and write it to your file
2. separate your strings of JSON with a delimiter, and then split them on read. Probably easiest to do that with newlines (ie, change f.write to f.puts and then on read iterate over each line and parse each line one at a time).