CODEWITHSUNDEEP
iterationjq
bluethundr
bluethundr

Reputation: 1345

JQ Pulling Inconsistent Results

If I run this very straight query on my json data from an aws command I get a correct result as to how many aws server instances I have in an account:

aws ec2 describe-instances | jq -r '.Reservations[].Instances[].InstanceId'

Produces a list of 47 instance IDs which corresponds to the number of server instances I have in the account. For example:

i-01adbf1408ef1a333
i-0f92d078ce975c138
i-0e4e117c44b17b417
and on up to 47 instances

This next query still produces the correct number of results:

aws ec2 describe-instances | jq -r '.Reservations[].Instances[] | [( .InstanceId ) ]'

However, if I add a query to include the name tags of the servers I get dramatically less number of server instances reported:

aws ec2 describe-instances | jq -r '.Reservations[].Instances[] | [( (.Tags[]|select(.Key=="Name")|.Value), .InstanceId ) ]'

This is the output of that command:

"i-08d3c05eed1316c9d"
"USAMZLAB10003","i-79eebb29"
"EOMLABAMZ1306","i-dbc98af4"
"USAMZLAB10002","i-d1dc1d83"
"i-0366c9bf18d27eb96"
"i-04d061334bc2f2d6b"
"USAMZLAB10007","i-f7a680a7"
"i-090e84eff4fece2b3"
"EOMLABAMZ1303","i-7cc98a53"
"EOMLABCSE713","i-08233926"
"i-0705eb3039cd56e04"
jq: error (at <stdin>:5013): Cannot iterate over null (null)

For some reason that query reports that there are only 11 aws server instances (when there should be 47). It does report that there are servers with and without name tags. But it's not reporting the correct number of servers.

It also produces the jq error "Cannot iterate over null".

I have put the original JSON into this paste:

Original JSON

How can I make the error more verbose so I can find out what's going on?

And why does adding the name tag to the query dramatically reduce the number of results?

Upvotes: 1

Views: 135

Answers (2)

peak
peak

Reputation: 116957

How can I make the error more verbose so I can find out what's going on?

You could use debug.

why does adding the name tag to the query dramatically reduce the number of results?

Because your jq program is at variance with your expectations; specifically, you have overlooked what happens when .Tags evaluates to null. To understand the mismatch, consider:

$ jq -n '{} | .Tags[]|select(.Key=="Name")|.Value'

Another issue is the handling of empty arrays. You might like to handle the case of empty arrays along the lines suggested by the following:

$ jq -n '{Tags: []} | (.Tags[] | select(.Key=="Name")|.Value) // null'
$ null

One solution

If you want null to appear whenever there isn't a tag:

.Reservations[].Instances[]
| [ ((.Tags // [])[] | select(.Key=="Name") | .Value) // null,
    .InstanceId  ]

Given your input, the first two lines of the output would be:

[null,"i-08d3c05eed1316c9d"]
["USAMZLAB10003","i-79eebb29"]

Variant using try

.Reservations[].Instances[]
| [ try (.Tags[] | select(.Key=="Name")|.Value) // null,
    .InstanceId  ]

Upvotes: 1

Jeff Mercado
Jeff Mercado

Reputation: 134571

In your json, not all instances have a set of Tags thus the error. You would have to handle it or substitute an empty array in its place with (.Tags // []). But overall, I would write it like this:

.Reservations[].Instances[] | [ (.Tags // [] | from_entries.Name), .InstanceId ]

Upvotes: 2

Related Questions