CODEWITHSUNDEEP
jsonbashjq
datacruncher
datacruncher

Reputation: 165

Merge multiple jq invocations to one

I am parsing a JSON file as follows and assigning it to the shell variables that I later test if it is not empty and taking according actions on top of it.

I am doing this using following jq commands:

 proxy_host=`jq -r .proxy_host <<< $line`
 proxy_port=`jq -r .proxy_port <<< $line`
 query_string=`jq -r .query_string <<< $line`
 realip_remote_addr=`jq -r .realip_remote_addr <<< $line`
 realpath_root=`jq -r .realpath_root <<< $line`
 request_body=`jq -r .request_body <<< $line`
 request_id=`jq -r .request_id <<< $line`
 request_method=`jq -r .request_method <<< $line`
 request_uri=`jq -r .request_uri <<< $line`
 scheme=`jq -r .scheme <<< $line`
 ssl_protocol=`jq -r .ssl_protocol <<< $line`
 ssl_ciphers=`jq -r .ssl_ciphers <<< $line`
 ssl_client_cert=`jq -r .ssl_client_cert <<< $line`
 ssl_client_fingerprint=`jq -r .ssl_client_fingerprint <<< $line`
 ssl_client_i_dn=`jq -r .ssl_client_i_dn <<< $line`
 ssl_client_raw_cert=`jq -r .ssl_client_raw_cert <<< $line`
 ssl_client_s_dn=`jq -r .ssl_client_s_dn <<< $line`
 ssl_client_serial=`jq -r .ssl_client_serial <<< $line`
 ssl_client_v_end=`jq -r .ssl_client_v_end <<< $line`
 ssl_client_v_remain=`jq -r .ssl_client_v_remain <<< $line`
 ssl_client_v_start=`jq -r .ssl_client_v_start <<< $line`
 ssl_client_verify=`jq -r .ssl_client_verify <<< $line`
 ssl_session_id=`jq -r .ssl_session_id <<< $line`
 tcpinfo_rtt=`jq -r .tcpinfo_rtt <<< $line`
 uri=`jq -r .uri <<< $line`

My json: 

{
  "host": "www.example.com",
  "hostname": "localhost",
  "proxy_add_x_forwarded_for": "127.0.0.1",
  "proxy_host": "",
  "proxy_port": "",
  "query_string": "",
  "realip_remote_addr": "8.1.2.3",
  "realpath_root": "/var/www/",
  "request_body": "",
  "request_id": "78e7cc17c207fc683992bae956150c4d",
  "request_method": "GET",
  "request_uri": "/",
  "scheme": "https",
  "server_name": "www.localhost.com",
  "ssl_protocol": "TLSv1.2",
  "ssl_ciphers": "AES128-GCM-SHA256:AES128-SHA:AES256-SHA:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:0x000a:0x00ff",
  "ssl_client_cert": "",
  "ssl_client_fingerprint": "",
  "ssl_client_i_dn": "",
  "ssl_client_raw_cert": "",
  "ssl_client_s_dn": "",
  "ssl_client_serial": "",
  "ssl_client_v_end": "",
  "ssl_client_v_remain": "",
  "ssl_client_v_start": "",
  "ssl_client_verify": "NONE",
  "ssl_session_id": "3e10c1253816aa4b3ea80df403e752fadaa1c6e532febd25e288acb4f5735617",
  "tcpinfo_rtt": "11142",
  "uri": "/index.html"
}

I do not want to use shell tools like awk to access the fields. Is there a way to simplify this using jq and bash not to have 30 independent jq invocations to get this done?

Upvotes: 3

Views: 317

Answers (2)

peak
peak

Reputation: 116740

Assuming the specified bash variable names are valid, the most robust method would be along these lines:

while IFS= read -d $'\0' -r line; do
    declare "$line"
done< <(data | jq -j -r 'to_entries[] | [.key, (.value | @sh)] | join("=") + "\u0000"' )

Notice the use of the jq filter @sh.

Upvotes: 1

Inian
Inian

Reputation: 85590

You obviously don't have do that many invocations of jq. Just do it once to dump all the key/value pairs and read them out in separate shell variables

#!/usr/bin/env bash

while IFS= read -r line; do
    declare "$line"
done< <(jq -r 'to_entries[] | [.key,.value] | join("=")' json)

Now you can print the variable from the command line having the same name as the one from the JSON key. To handle even more robust JSON strings that can contain literal newlines, emit the JSON output with a null-delimiter ("\u0000") and read it back.

Upvotes: 2

Related Questions