CODEWITHSUNDEEP
jsonmergestreamjq
Jiang Xiang
Jiang Xiang

Reputation: 3256

jq how to merge two JSON streams into a single JSON stream

How to merge two json streams/lists into a single stream of jsons like the following.

file 1:

{"key1": 1}
{"key1": 2}
{"key1": 3}

file 2:

{"key2": -1}
{"key2": -2}
{"key2": -3}

Expected output:

{"key1": 1, "key2": -1}
{"key1": 2, "key2": -2}
{"key1": 3, "key2": -3}

Upvotes: 1

Views: 401

Answers (1)

peak
peak

Reputation: 116870

Here is a "streaming" solution that is more efficient (both memory-wise and otherwise) than one that requires that both files be read in their entirety before producing any output:

< file1.json jq -nc --slurpfile file2 file2.json '
  # For each item $s in the stream s,
  # emit [$s, $t] where $t is the corresponding item in the input array
  def zip(s):
    . as $in
    | foreach s as $s (-1; .+1; [$s, $in[.]]);

  $file2 | zip(inputs) | add
'

If your jq does not support the --slurpfile command-line option, then use --argfile instead.

As a one-liner

foreach inputs as $s (-1; .+1; $s + $file2[.])

Upvotes: 4

Related Questions