CODEWITHSUNDEEP
bashshell
Dreams
Dreams

Reputation: 6122

Shell Script: Iterating over array of json

I have an array of jsons which I need to iterate over. I do a curl call and get this result and need to process it for something. The array looks like this: 

 [
{"id": "f0345a01", "name": "scala1"},
 {"id": "6f907cf", "name": "scala2"}, 
{"id": "d887f61", "name": "scala3"}, 
{"id": "5d07fca", "name": "scala5"}, 
{"id": "94ddaa", "name": "scala12"}
]

I need to get the id's from this array. I could not find any way to do so. I tried this following another stackoverflow question: 

for i in "${arr[@]}" 
do
    echo "$i"

done

Upvotes: 2

Views: 14420

Answers (5)

Todd A. Jacobs
Todd A. Jacobs

Reputation: 84443

Use a PCRE-Compatible Grep

If you don't want to do a full parse with jq (which is probably the right thing to do), then you can grep for your lines. There are all sorts of problems with this, though, including shell quoting issues, inconsistent whitespace in your corpus, no guarantee that you won't have nested objects, and so forth.

If you just want the id attributes, use a grep with PCRE support such as pcregrep. To keep the regular expression simple, you can then strip the surrounding quotes with tr. For example:

$ pcregrep -o '"id": \K"[[:xdigit:]]{7,8}"' <<< "$result" | tr -d \"
f0345a01
6f907cf
d887f61
5d07fca

Note that this is very dependent on your corpus, and on the presence of a non-standard grep, but it's still useful if you know your data and just want a list of IDs.

Upvotes: 2

NeronLeVelu
NeronLeVelu

Reputation: 10039

could use awk in this specific case (allowing other manip also if needed)

 YourCurlStream | awk 'NF>1{gsub( /[":,{}]/, "");$0=$0;print $2 }'

Upvotes: 1

hek2mgl
hek2mgl

Reputation: 158230

Use jq to obtain the ids:

curl http://... | jq -r '.[].id'

You can pipe that into a bash while loop if you want to perform further processing:

curl http://... | jq -r '.[].id' | while read id ; do
    do_something "${id}"
done

Upvotes: 10

Andriy Berestovskyy
Andriy Berestovskyy

Reputation: 8544

You might try the following "dirty hack":

ids=$(echo "${arr}" | tr -s '\n' ' ' | tr -s ',' '\n' \
    | grep '"id"\s*:' | cut -d '"' -f 4)
for i in ${ids}
do
    echo "===> $i"
done

As you can see it does not parse the array, but rather assumes the structure you provided and just grep/cut the array to get the ids.

  • tr -s '\n' ' ' substitutes all newlines to spaces, so it will correctly handle the case when "id" and the value are separated with a newline
  • tr -s ',' '\n' substitutes all commas to newlines, so we have one key-value per line
  • grep '"id"\s*:' greps all lines with "id" keys
  • finally cut -d '"' -f 4 uses quotation marks as a delimiter and cuts the value

Upvotes: 1

OrangesV
OrangesV

Reputation: 322

If you're curl does get you this formatted list

[
{"id": "f0345a01", "name": "scala1"},
{"id": "6f907cf", "name": "scala2"}, 
{"id": "d887f61", "name": "scala3"}, 
{"id": "5d07fca", "name": "scala5"}, 
{"id": "94ddaa", "name": "scala12"}
]

Then just doing something like:

curl -s "your_link" | awk '{print $2}' | tr -d '",'

OUTPUT:

cat sample | awk '{print $2}' | tr -d '","'

f0345a01
6f907cf
d887f61
5d07fca
94ddaa

Upvotes: 2

Related Questions