CODEWITHSUNDEEP
bashjq
zeroweb
zeroweb

Reputation: 2752

jq - return json as bash array

My question is similar to this question but I need something more to be done and can't quite figure out how to do.

This is my JSON string

{
  "value": "1"
}
{
  "value": "3"
}
{
  "value": "4"
}

I need to write a script in BASH to output

In the above example, 2 is missing from the json array. And the script should return 2. If 2 is present in the array it should return 5.

I think I may be able to write the logic to increment the number using a while loop in bash, but I am stuck at this point where I can't figure out how to convert this JSON string to bash array with just the value.

Below is the exact command that I used to get the JSON output. This is AWS CLI command to get all the instances from AWS that has a specific TAG. 

readarray -t arr < <(aws ec2 describe-instances --region=us-east-1 --filters --filters "Name=tag:NodeType,Values=worker" --query "Reservations[].Instances[].Tags[]" | jq -r '.[] | select(.Key == "NodeNumber") | {value: .Value}')
printf '%s\n' "${arr[@]}"

The above returns me 

{
  "value": "1"
}
{
  "value": "3"
}
{
  "value": "4"
}

However, I need to get just the VALUE field "value" as a bash array

Upvotes: 8

Views: 14295

Answers (4)

randomir
randomir

Reputation: 18697

To convert your JSON to a bash array, with help of jq:

$ readarray -t arr < <(jq '.value' file)
$ printf '%s\n' "${arr[@]}"
"1"
"3"
"4"

To fix your expanded example (the exact command), just don't use object construction {value: .Value}, but instead only .Value:

$ readarray -t arr < <(aws ec2 describe-instances --region=us-east-1 --filters --filters "Name=tag:NodeType,Values=worker" --query "Reservations[].Instances[].Tags[]" | jq -r '.[] | select(.Key == "NodeNumber") | .Value')
$ printf '%s\n' "${arr[@]}"
1
3
4

Notice the lack of double quotes, since the -r option now prints only raw string values, not raw JSON Objects.

After you get arr populated with values like this, you can easily iterate over it and perform tests, just as you described in your question.

Upvotes: 5

MauricioRobayo
MauricioRobayo

Reputation: 2356

This might give you some ideas:

#!/bin/bash

complete=true

while read value;do

    n=${n:-$value}

    if (( value != n ));then
            complete=false
            echo $n
            break
    fi

    let n++

done < <(jq -r '.value' json_file)
# or: done < <( command_that_outputs_json | jq -r '.value' )

$complete && echo $n

Upvotes: 0

Todd A. Jacobs
Todd A. Jacobs

Reputation: 84393

First, Store the Data

Given your raw data stored as a string in a json variable, perhaps with a here-document:

json=$(
    cat <<- EOF
        {
          "value": "1"
        }
        {
          "value": "3"
        }
        {
          "value": "4"
        }
EOF
)

Bash itself will do a reasonable job of prettifying it:

$ echo $json
{ "value": "1" } { "value": "3" } { "value": "4" }

Parsing the Data Into a Bash Array

There's more than one way to do this. Two of the more obvious ways are to use use jq or grep to extract the values into a Bash array with the shell's simple array notation:

values=( `echo $json | jq .value` )
echo "${values[@]}"
"1" "3" "4"

unset values
values=$(egrep -o '[[:digit:]]+' <<< "$json")
echo "${values[@]}"
1
3
4

There are certainly other ways to accomplish this task, but this seems like the low-hanging fruit. Your mileage may vary.

Caveat

The main thing to remember about Bash arrays is that they need to use expansions such as "${foo[@]}" when quoted, or ${bar[*]} when unquoted, if you want to use them for loops rather than indexing into them directly. Once they're in an array, though, you can access the entire array easily as long as you understand the different expansions and quoting rules.

Upvotes: 2

peak
peak

Reputation: 116870

Using the "-s" command-line option of jq, the following meets the requirements as I understand them, while generalizing them as well:

map(.value | tonumber)
| unique
| (1+max - min) as $length
| if $length > length then ([range(min;max+1)] - .)[]
  else max+1
  end

If you just want the least missing value, then replace the '[]' on the 'if' line by [0],

Upvotes: 1

Related Questions