CODEWITHSUNDEEP
jsonbashparameter-passingjqdata-processing
user3086917
user3086917

Reputation: 97

Passing JSON objects to jq arguments

Data sample - sample.json (full sample: https://pastebin.com/KFkVmc2M)

    {
      "ip": 3301234701,
      "_shodan": {
        "options": {
          "referrer": "7ae15507-f5cc-4353-b72e-5cc0b1c34c5e"
        },
      },
      "hash": -1056085507,
      "os": null,
      "title": "WHM Login",
      "opts": {
        "vulns": ["!CVE-2014-0160"],
        "heartbleed": "2017/08/29 09:57:30 196.196.216.13:2087 - SAFE\
        }
      },
      "isp": "Fiber Grid Inc",
      "http": {
        "redirects": [],
        "title": "WHM Login",
        "robots": null,
        "favicon": null,
        "host": "196.196.216.13",
        "html":
}

Script using jq which I hoped would work and haven't found a another solution yet.

   cat sample.json | jq \
   --arg key0   'Host' \
   --arg value0 '.host' \
   --arg key1   'Vulnerability' \
   --arg value1 '.opts.vulns[0]' \
   --arg key2   'ISP' \
   --arg value2 '.isp' \
   '. | .[$key0]= $value0 | .[$key1]=$value1 | .[$key2]=$value2' \
   <<<'{}'

The end result I hoped, but not getting:

{
  "Host": "196.196.216.13",
  "Vulnerability": "!CVE-2014-0160",
  "ISP": "Fiber Grid Inc"
}

Right now it just returns the object as a string and I've tried a lot of different ways to approach the problem. I am quite new to working with JSON and jq but based on what I've read so far, the solution might not be as simple as I'm wishing?

Simply put, why isn't the object being returned as a value of the sample.json object and what do I have to do, to make it work?

Thanks!

to: chepner

{
  "196.196.216.13":[
  "AS63119",
  "Fiber Grid Inc",
  "2017-08-29T06:57:22.546423",
  "!CVE-2014-0160"
],
"196.196.216.14":[
  "AS63119",
  "Fiber Grid Inc",
  "2017-08-29T06:57:22.546423",
  "!CVE-2014-0160"
]
}

Upvotes: 2

Views: 9322

Answers (3)

peak
peak

Reputation: 116957

Here's a variant solution using the approach suggested by @CharlesDuffy:

$ config='{"Host": ["http", "host"], 
           "Vulnerability": ["opts", "vulns", 0], 
           "ISP": ["isp"]}'
$ jq --argjson config "$config" '
   . as $in
   | $config
   | map_values( . as $p | $in | getpath($p) )' pastebin.json

Output:

{
  "Host": "196.196.216.13",
  "Vulnerability": "!CVE-2014-0160",
  "ISP": "Fiber Grid Inc"
}

This incidentally highlights @chepner's point: the care and maintenance required for $config is no less than the care and maintenance required for the corresponding jq query, whereas the jq query language -- by design -- offers far more flexibility.

Upvotes: 4

peak
peak

Reputation: 116957

In response to the supplementary question ("to: chepner"):

The pastebin data does not contain "196.196.216.14" so the supplementary question is unclear, but you will probably want to start with: 

jq '{(.ip_str): [.asn, .isp, .timestamp, opts.vulns[0] ]}' pastebin.json

This produces:

{
  "196.196.216.13": [
    "AS63119",
    "Fiber Grid Inc",
    "2017-08-29T06:57:22.546423",
    "!CVE-2014-0160"
  ]
}

The key point here is the pair of parentheses around .ip_str.

As mentioned elsewhere, the jq query is again so simple and straightforward that it seems doubtful that a parameterized version will offer any advantage.

Upvotes: 1

peak
peak

Reputation: 116957

  1. jq does not support the evaluation of jq expressions in the way that would be required for your attempt to work. You could do some kind of shell interpolation, but it would be better to use JSON paths, e.g. rather than --arg value0 .host, you could write --arg value0 "host", etc. In the following, I've used getpath/1.

  2. There is no need to prepend the jq filter with '. |'

Anyway, assuming the JSON contents of the pastebin are in pastebin.json, you could write:

  jq \
   --arg key0        Host \
   --argjson value0  '["http","host"]' \
   --arg key1        Vulnerability \
   --argjson value1  '["opts", "vulns", 0]' \
   --arg key2        ISP \
   --argjson value2  '["isp"]' \
   '. as $in
    | {}
    | .[$key0] = ($in|getpath($value0))
    | .[$key1] = ($in|getpath($value1))
    | .[$key2] = ($in|getpath($value2))' \
   pastebin.json

This would have the result:

{
  "Host": "196.196.216.13",
  "Vulnerability": "!CVE-2014-0160",
  "ISP": "Fiber Grid Inc"
}

Upvotes: 2

Related Questions