CODEWITHSUNDEEP
jsonbashjq
Alex
Alex

Reputation: 2805

Append new keypair to JSON file with jq

I've read almost every question I can find on SO with the same general premise and I've gotten close three different times, but I just can't make this work.

Given a JSON file called parameters.json with keypairs like so; (shortened for sharing purposes)

[
  {
    "ParameterKey": "Foo1",
    "ParameterValue": "Bar1"
  },
  {
    "ParameterKey": "Foo2",  
    "ParameterValue": "Bar2"
  }
]

I want to add another keypair, say

  {
    "ParameterKey": "Foo3",
    "ParameterValue": "Bar3"
  }

So I end up with a file like

[
  {
    "ParameterKey": "Foo1",
    "ParameterValue": "Bar1"
  },
  {
    "ParameterKey": "Foo2",  
    "ParameterValue": "Bar2"
  },
  {
    "ParameterKey": "Foo3",
    "ParameterValue": "Bar3"
  }
]

Use Case & Attempts

I'm writing a bash script that copies and updates parameter files for AWS - this is where the keypairs come in. We have a new keypair that needs to be added to any existing files when they're copied. I can get the if/then done fine, it's physically adding the new keypair that is stumping me.

With a ton of googling I ended up with this clunky workaround, which I don't really understand and, while it generates the keypair just fine, doesn't add it back into parameters.json

item='{"ParameterKey": "RTSMSnapshotID"}'

jq --argjson item "$item" '$item + {"ParameterValue": ""}' parameters.json

I get the keypair printed back to me correctly on the command line, so I figure add | sponge parameters.json but that then gives me an empty file.

I also considered a sed workaround that trimmed the last line of the file and then appended the new keypair (and EOF) but of course sed doesn't play well with newlines and I can't get it to work.

I'm open to solutions that use anything in bash and anything in jq. I am still a bash/jq noob so detailed explanations are appreciated.

Upvotes: 1

Views: 5109

Answers (2)

peak
peak

Reputation: 116967

If you have one file with a JSON array (say parameters.json), and one or more additional files (say file*.json) with JSON entities that you want to append to the array in the first file, and if you want to overwrite the first file, here's a rather sly solution, which however requires jq 1.5 or later:

$ jq 'reduce inputs as $in (.; . + [$in])' parameters.json file*.json |
    sponge parameters.json

Note that using this approach, each top-level JSON entity in each of the auxiliary files will be appended to the array separately.

What makes this a bit crafty is that inputs is being used here without the -n option.

Since you mentioned bash, please note that if any of the "file*.json" sources is a process, you could use <( ... ) rather than specifying a filename, e.g.:

$ jq 'reduce inputs as $in (.; . + [$in])' parameters.json <(curl -Ss ...) |
    sponge parameters.json

If you don't have sponge, then you can use the idiom:

jq ..... > OUT && mv IN OUT

Upvotes: 1

Inian
Inian

Reputation: 85865

The idea of using argjson is right, but the syntax is as below on jq-1.5

jq --argjson obj '{ "ParameterKey": "Foo3", "ParameterValue": "Bar3" }' '. + [$obj]' < json 
[
  {
    "ParameterKey": "Foo1",
    "ParameterValue": "Bar1"
  },
  {
    "ParameterKey": "Foo2",
    "ParameterValue": "Bar2"
  },
  {
    "ParameterKey": "Foo3",
    "ParameterValue": "Bar3"
  }
]

Check the jq-documentation for more information about the + and += operators.

Am not aware of ways to do this in jq itself, but you can use a neat bash trick to do this,

jq --argjson obj '{ "ParameterKey": "Foo3", "ParameterValue": "Bar3" }' '. + [$obj]' < json > temp && mv temp json

Upvotes: 4

Related Questions