CODEWITHSUNDEEP
jsonbashjq
Akshatha Nadig
Akshatha Nadig

Reputation: 21

Updating JSON object values dynamically and storing them in a new file in BASH

I am completely new (an intern with less than two weeks of hands-on experience) to BASH, JSON and jq. But I have been given a task to replace JSON object value from an array iteratively. I have managed to write the following code.

I am trying to replace the values of the key, "name", in the json file incrementally to the names in the array stored in another text file. Basically just updating the names of the nodes.

Here is a snippet of the JSON file to be replaced. The whole file is way too big to be posted here. But the path to the key "name" is ".lab.racks[].nodes[].name"

    "roles": [
           "bla bla",
           "bla bla"
      ],
      "name": "Node1",
       "power": {
       "address": "10.182.149.145",
       "type": "bla bla",
       "user": "bla bla",
       "pass": "bla bla "
       },

The "name": "Node1" must be replaced as "name": "Tom-cat". The name Tom-cat is dynamically generated and changes everytime a Metal as a Service (MaaS) script is run. This name "Tom-cat" and all other new names generated by MaaS is cut (using awk) and stored in a text file newhostnames.txt

The textfile looks like this 

#newhostnames.txt
Tom-cat
Lucky-worm
Wom-bat

So the goal is to replace the "name" key from demolabconfig.json with the names stored in the text file.

"name": "Node1" must be replaced as "name":"Tom-cat"
];
.....
.....
],
"name": "Node2" must be replaced as "name":"Lucky-worm"
];
.....
.....
],
"name": "Node3" must be replaced as "name":"Wom-bat"

The indexing is done for the key "nodes": .lab.racks[].nodes[$i].name

The code:

readarray -t array < newhostnames.txt
array=("${array[@]:1}")
array_length=${#array[@]}
for((i=0;i<${array_length};i++));
do
    declare -x  NEW_NODENAME
    OLD_NODENAME=$(jq -r ".lab.racks[].nodes[$i].name" demolabconfig.json)
    echo "$OLD_NODENAME"
    NEW_NODENAME="${array[i]}"
    echo "$NEW_NODENAME"
    jq ".lab.racks[].nodes[$i].name=env.NEW_NODENAME" demolabconfig.json > newdemolabconfig.json
done

But the code only replaces one value in the end for the last $i i.e, the last key_value pair. All other preceding nodes retain the same name as in the original JSON file.

I have tried by using an if statement to break the loop if the value is already updated so it is not re_written from the old demolabconfig.json file. But that is also not working!

if [[ "$OLD_NODENAME" -ne "$NEWNODENAME" ]]; then
    jq ".lab.racks[].nodes[$i].name=env.NEW_NODENAME" demolabconfig.json > newdemolabconfig.json
else
    break
fi

This if loop was written with a while loop instead of for. This replaces the last name with null.

Please suggest how I can fix this error and improve my code. Thanks :)

Upvotes: 2

Views: 1754

Answers (2)

peak
peak

Reputation: 116957

The goal here should be to call jq as few times as possible. Accordingly, we present a solution which involves just two calls to jq, irrespective of the number of "names" in newhostnames.txt.

First let's suppose input.json contains the following:

{"nodes":[{"name":"old1"},{"name":"old2"},{"name":"old3"}]}

Let's also suppose the follow jq program is in the file program.jq:

  reduce range(0; $newhostnames | length) as $i (.;
    .nodes[$i].name = $newhostnames[$i])

Then the invocation:

jq -c --slurpfile newhostnames <(jq -nR inputs newhostnames.txt ) \
   -f program.jq input.json

produces:

{"nodes":[{"name":"Tom-cat"},{"name":"Lucky-worm"},{"name":"Wom-bat"}]}

(The -c command-line option produces compressed output.)

If your shell does not support the above invocation, you could (for example) put the output of jq -nR inputs newhostnames.txt in a temporary file.

Of course the subsidiary task of converting the .txt file into a JSON array or a stream of JSON strings can be accomplished in other ways.

Robustification

If there's a chance that newhostnames.txt contains unwanted blank lines, then one way to skip them would be to add one line to program.jq so that it would look like this:

($newhostnames | map(select(length>0))) as $newhostnames
| reduce range(0; $newhostnames | length) as $i (.;
      .nodes[$i].name = $newhostnames[$i])

Notice that the $-variable name can be reused.

Upvotes: 1

chepner
chepner

Reputation: 532112

The biggest problem is that you are overwriting your modified file using a copy of the original file at each step. You want the new file to be the input for the next iteration of the loop:

{
   read   # discard the first line
   readarray -t array
} < newhostnames.txt

cp demolabconfig.json newdemolabconfig.json
input=newdemolabconfig.json

for((i=0;i<${#array[@]};i++));
do
    old_nodename=$(jq -r ".lab.racks[].nodes[$i].name" "$input")
    new_nodename="${array[i]}"
    if [[ $old_nodename == $new_nodename ]]; then
        continue
    fi
    jq ".lab.racks[].nodes[$i].name=env.NEW_NODENAME" "$input" > tmp.json
    mv tmp.json "$input"
done

Upvotes: 0

Related Questions