CODEWITHSUNDEEP
jsonbashjq
Goddy
Goddy

Reputation: 44

JQ - flatten out / unroll double-nested objects with different key names

Having the following json structure:

{
    "outerDescription1": {
        "innerDescription1": {
            "otherProperties": 1,
            "items": [
                "arrayItem1",
                "arrayItem2"
            ]
        }
    },
    "outerDescription2": {
        "innerDescription2": {
            "otherProperties": 2,
            "items": [
                "arrayItem3",
                "arrayItem4"
            ]
        }
    }
}

I would like to get the following result:

{
    "item": "arrayItem1",
    "outer": "outerDescription1",
    "inner": "innerDescription1",
    "otherProperties": 1
}
{
    "item": "arrayItem2",
    "outer": "outerDescription1",
    "inner": "innerDescription1",
    "otherProperties": 1
}
{
    "item": "arrayItem3",
    "outer": "outerDescription2",
    "inner": "innerDescription2",
    "otherProperties": 2
}
{
    "item": "arrayItem4",
    "outer": "outerDescription2",
    "inner": "innerDescription2",
    "otherProperties": 2
}

Assumption: there are many outerDescription and innerDescription keys and they are not known upfront.

Single level unroll is would be simple, but unrolling double-nested objects with different keys is a challenge to me.

The closest what I was able to get was:

jq "with_entries(.value = {outer: .key} + .value)[]"

which resulted in:

{                              
  "outer": "outerDescription1",
  "innerDescription1": {       
    "otherProperties": 1,      
    "items": [                 
      "arrayItem1",            
      "arrayItem2"             
    ]                          
  }                            
}                              
{                              
  "outer": "outerDescription2",
  "innerDescription2": {       
    "otherProperties": 2,      
    "items": [                 
      "arrayItem3",            
      "arrayItem4"             
    ]                          
  }                            
}

But right now, without knowing the next nested key name exactly, I am not able to unroll for a second time in the same way as outer would be swallowed.

I am using JQ 1.6

Upvotes: 0

Views: 748

Answers (3)

peak
peak

Reputation: 116919

Here's a straightforward solution using to_entries:

  to_entries[]
  | .key as $outer
  | .value
  | to_entries[]
  | .key as $inner
  | .value
  | .otherProperties as $otherProperties
  | .items[]
  | {item: ., $outer, $inner, $otherProperties}

Or, equivalently but without variables:

  to_entries[]
  | {outer: .key} +
    ( .value | to_entries[] | {inner: .key} +
    ( .value | {otherProperties} +
    ( .items[] | {item: .} )))

If the order of keys is important, you could (for example) add an expression such as {item, outer, inner, otherProperties} .

Addendum: To retain all the keys at the "otherProperties" level (apart from .items), you could tweak the solution above:

  to_entries[]
  | {key} +
    ( .value | to_entries[] | {key} +
    ( .value | del(.items) +
    (.items[] | {item: .} )))

Upvotes: 1

knittl
knittl

Reputation: 265668

If you know the name of your items property, the following equivalent programs are quite readable:

to_entries[]
| { outer: .key, nested: (.value | to_entries[]) }
| { item: .nested.value.items[] }
+ { outer, inner: .nested.key }
+ (.nested.value | del(.items))

to_entries[]
| { outer: .key, nested: (.value | to_entries[]) }
| { item: .nested.value.items[], outer, inner: .nested.key }
+ (.nested.value | del(.items))

to_entries[]
| { outer: .key, nested: (.value | to_entries[]) }
| { outer }
+ (.nested
    | { inner: .key, item: .value.items[] }
    + (.value | del(.items))
)

to_entries[]
| { outer: .key, nested: (.value | to_entries[]) }
| { outer }
+ (.nested
    | { inner: .key }
    + (.value | { item: .items[] } + del(.items))
)

You could also use a function:

def merge(name; field): { (name): field } + del(field);

to_entries[]
| { outer: .key, nested: (.value | to_entries[]) }
| { outer }
+ (.nested
    | { inner: .key }
    + (.value | merge("item"; .items[]))
)

JSON does not care about key order in objects, but if – for whatever reason – you need to force a specific order, you append a final transformation. But this requires to list all keys again:

...
| { item, outer, inner, otherProperties } # force key/property order

Output:

{
  "item": "arrayItem1",
  "outer": "outerDescription1",
  "inner": "innerDescription1",
  "otherProperties": 1
}
{
  "item": "arrayItem2",
  "outer": "outerDescription1",
  "inner": "innerDescription1",
  "otherProperties": 1
}
{
  "item": "arrayItem3",
  "outer": "outerDescription2",
  "inner": "innerDescription2",
  "otherProperties": 2
}
{
  "item": "arrayItem4",
  "outer": "outerDescription2",
  "inner": "innerDescription2",
  "otherProperties": 2
}

Upvotes: 1

pmf
pmf

Reputation: 36326

You could use path to find the according paths, and getpath to retrieve their values:

path(.[][].items[]) as $p
| {item: getpath($p), outer: $p[0], inner: $p[1]}
+ (getpath($p[:-2]) | del(.items))

{
  "item": "arrayItem1",
  "outer": "outerDescription1",
  "inner": "innerDescription1",
  "otherProperties": 1
}
{
  "item": "arrayItem2",
  "outer": "outerDescription1",
  "inner": "innerDescription1",
  "otherProperties": 1
}
{
  "item": "arrayItem3",
  "outer": "outerDescription2",
  "inner": "innerDescription2",
  "otherProperties": 2
}
{
  "item": "arrayItem4",
  "outer": "outerDescription2",
  "inner": "innerDescription2",
  "otherProperties": 2
}

Demo

Upvotes: 2

Related Questions