CODEWITHSUNDEEP
bashjq
Gowtham Ramamoorthy
Gowtham Ramamoorthy

Reputation: 896

cannot call bash environment variable inside jq

In the below script, I am not able to successfully call the "repovar" variable in the jq command.

cat quayrepo.txt | while read line
do
  export repovar="$line"
  jq -r --arg repovar "$repovar" '.data.Layer| .Features[] | "\(.Name), \(.Version), $repovar"' severity.json > volume.csv
done

The script uses a text file to loop through the repo names

quayrepo.txt---> file has the list of names in this case the file has a value of "Reponame1"

sample input severity.json file:

{
  "status": "scanned",
  "data": {
    "Layer": {
      "IndexedByVersion": 3,
      "Features": [
        {
          "Name": "elfutils",
          "Version": "0.168-1",
          "Vulnerabilities": [
            {
              "NamespaceName": "debian:9",
              "Severity": "Medium",
              "Name": "CVE-2016-2779"
            }
          ]
        }
      ]
    }
  }
}

desired output: 

elfutils, 0.168-1, Medium, Reponame1

Required output: I need to retrieve the value of my environment variable as the last column in my output csv file

Upvotes: 0

Views: 281

Answers (3)

Charles Duffy
Charles Duffy

Reputation: 295272

You don't need a while read loop in bash at all; jq itself can loop over your input lines, even when they aren't JSON, letting you run jq only once, not once per line in quayrepo.txt.

jq -rR --slurpfile inJson severity.json <quayrepo.txt >volume.csv '
($inJson[0].data.Layer | .Features[]) as $features |
[$features.Name, $features.Version, .] |
@csv
'
  • jq -R specifies raw input, letting jq directly read lines from quayrepo.txt into .
  • jq --slurpfile varname filename.json reads filename.json into an array of JSON objects parsed from that file. If the file contains only one object, one needs to refer to $varname[0] to refer to it.
  • @csv converts an array to a CSV output line, correctly handling data with embedded quotes or other oddities that require special processing.

Upvotes: 1

thehole
thehole

Reputation: 540

#!/usr/bin/env bash

while read line
do
  jq -r --arg repovar "$line" '.data.Layer.Features[] | .Name + ", " + .Version + ", " + $repovar' severity.json
done < quayrepo.txt > volume.csv

with quayrepo.txt as 

Reponame1

and severity.json as 

{
  "status": "scanned",
  "data": {
    "Layer": {
      "IndexedByVersion": 3,
      "Features": [
        {
          "Name": "elfutils",
          "Version": "0.168-1",
          "Vulnerabilities": [
            {
              "NamespaceName": "debian:9",
              "Severity": "Medium",
              "Name": "CVE-2016-2779"
            }
          ]
        }
      ]
    }
  }
}

produces volume.csv containing 

elfutils, 0.168-1, Reponame1

To @peak's point, changing > to >> in ...severity.json >> volume.csv will create a multi-line csv instead of just overwriting until the last line

Upvotes: 1

LMC
LMC

Reputation: 12662

You need to surround $repovar with parenthesis, as the other values

repovar='qweqe'; jq -r --arg repovar "$repovar" '.data.Layer| .Features[] | "\(.Name), \(.Version), \($repovar)"' tmp.json

Result:

elfutils, 0.168-1, qweqe

There's no need for the export.

Upvotes: 4

Related Questions