Merge two complex JSON objects with arrays

Question

I have the two following json as input:

{
  "one": {
    "vars": [
      {
        "name": "a",
        "value": "a"
      },
      {
        "name": "b",
        "value": "b"
      }
    ]
  },
  "two": {
    "vars": [
      {
        "name": "c",
        "value": "c"
      },
      {
        "name": "d",
        "value": "d"
      }
    ]
  },
  "extras": "whatever"
}

{
  "one": {
    "vars": [
      {
        "name": "e",
        "value": "e"
      },
      {
        "name": "f",
        "value": "f"
      }
    ]
  },
  "two": {
    "vars": [
      {
        "name": "g",
        "value": "g"
      },
      {
        "name": "h",
        "value": "h"
      }
    ]
  }
}

And I'd like to merge them in order to obtain the following result where each of the vars array of each section are merged together:

{
  "one": {
    "vars": [
      {
        "name": "a",
        "value": "a"
      },
      {
        "name": "b",
        "value": "b"
      },
      {
        "name": "e",
        "value": "e"
      },
      {
        "name": "f",
        "value": "f"
      }
    ]
  },
  "two": {
    "vars": [
      {
        "name": "c",
        "value": "c"
      },
      {
        "name": "d",
        "value": "d"
      },
      {
        "name": "g",
        "value": "g"
      },
      {
        "name": "h",
        "value": "h"
      }
    ]
  },
  "extras": "whatever"
}

Ideally but not mandatory:

• the keys (here one and two) would be arbitrary and an undefined number of them could be present.
• the vars array would not contain duplicate (based on name) and right precedence would be applied to override values from the first array.

I managed to merge the two objects and only 1 array with the following command but the key is hardcoded and I'm a bit stuck from there:

jq -s '.[0].one.vars=([.[].one.vars]|flatten)|.[0]' file1.json file2.json

Answer 1

First, here is a solution which is oblivious to the top-level key names, but which does not attempt to avoid duplicates:

$A
| reduce keys_unsorted[] as $k (.;
    if .[$k] | (type == "object") and has("vars")
    then (.[$k]|.vars) += ($B[$k]|.vars) else . end )

Here of course $A and $B refer to the two objects. You can set $A and $B in several ways.

If you want to reorder the top-level keys, you can simply extend the above with a filter specifying the order, e.g.: {extras, two, one}.

To avoid duplicates, I'd suggest writing a helper function to do just that, as illustrated in the following section.

Avoiding duplicates

def extend(stream):
  reduce stream as $s (.;
    (map(.name) | index($s|.name)) as $i
    | if $i then .[$i] += $s
      else . + [$s]
      end) ;


$A
| reduce keys_unsorted[] as $k (.;
    if .[$k] | (type == "object") and has("vars")
    then (.[$k].vars) = ( .[$k].vars | extend(($B[$k].vars[])))
    else . end
  )

Answer 2

jq -n 'input as $b | input
| .one.vars |= . + $b.one.vars
| .two.vars |= . + $b.two.vars' file2.json file1.json

file1.json must come after file2.json in order to preserve extras.