CODEWITHSUNDEEP
jsonlinuxshelljq
Graham Nicholls
Graham Nicholls

Reputation: 561

Json extracting list of field names for a key

jq is a great tool. If I have a json file, I can extract the list of keys

like so:

jq keys filename | tr -d '][",'

eg:

$jq  keys pipeline/components/devsvr.json  | tr -d '][",'
server1
server2

Now I'd like to iterate over each key, listing the top-level field names only (I'm trying to write a very simple validator for some CloudFormation json files).

The shell part is easy - just a for loop, but I can't work out how to say to jq: "show me all the field-names only for key x.

I'll then (in shell) check that each field I require is present. As an aside, jq will report with an error if the json is malformed, which is useful as well.

Here's an example file:

{
  "server1": {
    "type": "single-instance",
    "stage": "10default",
    "descriptor": {
      "Resources": {
        "Instance": {
          "Properties": {
            "InstanceType": "t2.medium",
            "ImageId": {
              }
            }
          },
          "Metadata": {
            "AWS::CloudFormation::Init": {
              "app": {
                "packages": {
                  "yum": {
                    "tmux": [],
                    "vim": []
                  }
                },
                "files": {
                "sources": {},
                "commands": {},
                "services": {}
              }
            }
          }
        }
      }
    }
  },
  "server2": {
    "type": "single-instance",
    "stage": "10default",
    "descriptor": {
      "Resources": {
        "Instance": {
          "Properties": {
            "InstanceType": "t2.medium",
            "ImageId": {
              }
            }
          },
          "Metadata": {
            "AWS::CloudFormation::Init": {
              "app": {
                "packages": {
                  "yum": {
                    "tmux": [],
                    "vim": []
                  }
                },
                "files": {
                "sources": {},
                "commands": {},
                "services": {}
              }
            }
          }
        }
      }
    }
  }
}

So with the example above, I'd run jq keys to get the list of keys, then iterate over that. I've possibly answered my own question with the sed example which I've commented with, but that's a bit heath-robinson, IMO. Here's the result:

$jq  '.server1' /tmp/afile   | sed -n 's#^  \"\([^"]*\).*$#\1#p'
type
stage
descriptor

(that would be in a loop around the output of the jq 'keys' command).

Update: As per @peak and @Aaron, I can do this:

jq -r '. as $in | keys[] | . as $serverName | $in[$serverName] | keys | join(",") | "\($serverName) : \(.)"' pipeline/components/devsvr.json
devsvr : descriptor,stage,type
devsvr1 :

With the proviso that http://json-schema.org exists for a more rigorous implementation.

Upvotes: 0

Views: 937

Answers (1)

peak
peak

Reputation: 116977

... using as small a toolset as possible.

As far as processing the JSON is concerned, it seems you need look no further than jq.

For example, your first command can be simplified to:

jq -r 'keys[]' devsvr.json

Use keys_unsorted if you want the keys in the original order.

More importantly, iterating over the keys can (and probably should) be done in jq. I'm not sure what your exact requirements are but you should easily be able to adapt the following:

jq -r '.[] | keys_unsorted[]' devsvr.json

which produces:

type
stage
descriptor
type
stage
descriptor

Update

In accordance with the update to the Q, the following filter should be considered:

 keys[] as $serverName 
 | .[$serverName] 
 | "\($serverName) : \(keys|join(","))"

Upvotes: 1

Related Questions