CODEWITHSUNDEEP
jsonmergejq
wsa
wsa

Reputation: 123

Update one JSON file values with values from another JSON using JQ (on all levels)

I have two JSON files:

source.json:

{
  "general": {
    "level1": {
      "key1": "x-x-x-x-x-x-x-x",
      "key3": "z-z-z-z-z-z-z-z",
      "key4": "w-w-w-w-w-w-w-w"
    },
    "another" : {
      "key": "123456",
      "comments": {
        "one": "111",
        "other": "222"
      }
    }
  },
  "title": "The best"
}

and the

target.json:

{
  "general": {
    "level1": {
      "key1": "xxxxxxxx",
      "key2": "yyyyyyyy",
      "key3": "zzzzzzzz"
    },
    "onemore": {
      "kkeeyy": "0000000"
    }
  },
  "specific": {
    "stuff": "test"
  },
  "title": {
    "one": "one title",
    "other": "other title"
  }
}

I need all the values for keys which exist in both files, copied from source.json to target.json, considering all the levels.
I've seen and tested the solution from this post. It only copies the first level of keys, and I couldn't get it to do what I need. The result from solution in this post, looks like this:

{
  "general": {
    "level1": {
      "key1": "x-x-x-x-x-x-x-x",
      "key3": "z-z-z-z-z-z-z-z",
      "key4": "w-w-w-w-w-w-w-w"
    },
    "another": {
      "key": "123456",
      "comments": {
        "one": "111",
        "other": "222"
      }
    }
  },
  "specific": {
    "stuff": "test"
  },
  "title": "The best"
}

Everything under the "general" key was copied as is.
What I need, is this:

{
  "general": {
    "level1": {
      "key1": "x-x-x-x-x-x-x-x",
      "key2": "yyyyyyyy",
      "key3": "z-z-z-z-z-z-z-z"
    },
    "onemore": {
      "kkeeyy": "0000000"
    }
  },
  "specific": {
    "stuff": "test"
  },
  "title": {
    "one": "one title",
    "other": "other title"
  }
}

Only "key1" and "key3" should be copied.
Keys in target JSON must not be deleted and new keys should not be created.

Can anyone help?

Upvotes: 2

Views: 1140

Answers (3)

peak
peak

Reputation: 116957

The following provides a solution to the revised question, which is actually about "paths" rather than "keys".

([$target|paths(scalars)] | unique) as $paths
| reduce ($source|paths(scalars)) as $p
    ($target;
     if $paths | bsearch($p) > -1 
     then setpath($p; $source|getpath($p))
     else . end)

unique is called so that binary search can be used subsequently.

Invocation:

jq -n --argfile source source.json --argfile target target.json -f program.jq

Upvotes: 0

peak
peak

Reputation: 116957

[Note: this response answers the original question, with respect to the original data. The OP may have had paths in mind rather than keys.]

There is no need to compute the intersection to achieve a reasonably efficient solution.

First, let's hypothesize the following invocation of jq:

jq -n --argfile source source.json --argfile target target.json -f copy.jq

In the file copy.jq, we can begin by defining a helper function:

# emit an array of the distinct terminal keys in the input entity
def keys: [paths | .[-1] | select(type=="string")] | unique;

In order to inspect all the paths to leaf elements of $source, we can use tostream:

($target | keys) as $t
| reduce ($source|tostream|select(length==2)) as [$p,$v]
    ($target;
     if $t|index($p[-1]) then setpath($p; $v) else . end)

Alternatives

Since $t is sorted, it would (at least in theory) make sense to use bsearch instead of index:

 bsearch($p[-1]) > -1

Also, instead of tostream we could use paths(scalars).

Putting these alternatives together:

($target | keys) as $t
| reduce ($source|paths(scalars)) as $p
    ($target;
     if $t|bsearch($p[-1]) > -1 
     then setpath($p; $source|getpath($p))
     else . end)

Output

{
  "general": {
    "level1": {
      "key1": "x-x-x-x-x-x-x-x",
      "key2": "yyyyyyyy",
      "key3": "z-z-z-z-z-z-z-z"
    },
    "onemore": {
      "kkeeyy": "0000000"
    }
  },
  "specific": {
    "stuff": "test"
  }
}

Upvotes: 1

Jeff Mercado
Jeff Mercado

Reputation: 134571

One approach you could take is get all the paths to all scalar values for each input and take the set intersections. Then copy values from source to target from those paths.

First we'll need an intersect function (which was surprisingly difficult to craft):

def set_intersect($other):
    (map({ ($other[] | tojson): true }) | add) as $o
    | reduce (.[] | tojson) as $v ({}; if $o[$v] then .[$v] = true else . end)
    | keys_unsorted
    | map(fromjson);

Then to do the update:

$ jq --argfile s source.json '
reduce ([paths(scalars)] | set_intersect([$s | paths(scalars)])[]) as $p (.;
    setpath($p; $s | getpath($p))
)
' target.json

Upvotes: 1

Related Questions