CODEWITHSUNDEEP
jsonmergekeyjq
Renee Cooper
Renee Cooper

Reputation: 23

jq: How can I combine data from duplicate keys

I have a fairly complex JSON data structure that I've managed to use jq to filter down to certain keys and their values. I need to combine the results though, so duplicate keys have only one array of values.
e.g.

     {
    "1.NBT.B": [
      {
        "id": 545
      },
      {
        "id": 546
       }
    ]
  },
  {
    "1.NBT.B": [
      {
        "id": 1281
      },
      {
        "id": 1077
      }
    ]
  }

would result in 

 {
    "1.NBT.B": [
      {
        "id": 545
      },
      {
        "id": 546
       },
      {
        "id": 1281
      },
      {
        "id": 1077
      }
    ]
  },
...

or even better:

[{"1.NBT.B": [545, 546, 1281, 1077]}, ...]

I need to do it without having to put in the key ("1.NBT.B") directly, since there are hundreds of these keys. I think what has me most stuck is that the objects here aren't named -- the keys are not the same between objects.

Something like this only gives me the 2nd set of ids, completing skipping the first:

reduce .[] as $item ({}; . + $item)

Upvotes: 2

Views: 2921

Answers (1)

peak
peak

Reputation: 116740

Part 1

The following jq function combines an array of objects in the manner envisioned by the first part of the question.

# Given an array of objects, produce a single object with an array at
# every key, the array at each key, k, being formed from all the values at k.
def merge:
  reduce .[] as $o ({}; reduce ($o|keys)[] as $key (.; .[$key] += $o[$key] ));

With this definition together with the line:

merge

in a file, and with the example input modified to be a valid JSON array, the result is:

{
  "1.NBT.B": [
    {
      "id": 545
    },
    {
      "id": 546
    },
    {
      "id": 1281
    },
    {
      "id": 1077
    }
  ]
}

Part 2

With merge as defined above, the filter:

merge | with_entries( .value |= map(.id) )

produces:

{
  "1.NBT.B": [
    545,
    546,
    1281,
    1077
  ]
}

Upvotes: 3

Related Questions