CODEWITHSUNDEEP
pythonjsonlinux
J.Doe
J.Doe

Reputation: 413

Parsing input argv json string

I want to parse the output of a linux jq call into a python script that will decode the output jq json str and do things with it.

The output of my jq call jq '.geometry' myJson.json is something like this:

{
  "coordinates": [
    [
      [
        5,
        2
      ],
      [
        5.4,
        3
      ],
      [
        3,
        2.1
      ]
    ]
  ],
  "crs": {
    "properties": {
      "name": "foo"
    },
    "type": "name"
  },
  "type": "Polygon"
}

I have written a small python executable that decodes the output json string into python objects and then do things:

import collections
import json
import sys
import logging

if __name__ == '__main__':

    try:
        geoJsonStr = str(sys.argv[1:])
        print geoJsonStr ## This for some reason only prints an empty slice '[]'
        data = json.loads(geoJsonStr)
        coordinates = data['coordinates'] ## TypeError: list indices must be integers, not str
        ## Do things here

    except ValueError as e:
        logging.error(e.message)
        exit(1)

This is how I try to call it:

jq '.geometry' geoJson.json | myPythonProgram

However I am getting some python errors as noted in my code snippet above. I'm thinking it's the way that I am passing the jq output into my python executable. Somehow the entire json string isn't being picked up as an argv argument.

My first error is the print GeoJsonStr prints out an empty [] slice from argv[1:]. So I am probably incorrectly passing the json string into the python script. The subsequent error is:

coordinates = data['coordinates']

TypeError: list indices must be integers, not str

This is probably more or less because there was nothing to decode.

Upvotes: 1

Views: 2514

Answers (1)

Seth
Seth

Reputation: 46433

When you use a pipe to send data to a program, you can access the data via stdin, not with as an argument in argv.

For example, suppose you had the following program:

foo.py: 

import sys
data = sys.stdin.read()
print "I got", len(data), "characters!"

Piping some data into it would give you output like this: 

$ echo "foobar" | python foo.py
I got 6 characters!

Note that in this example, the call to python contains an argument (foo.py) that is completely separate from the input.

In your particular case you could read stdin directly like the above example, or pass sys.stdin directly as an argument to json.load:

import sys
...
obj = json.load(sys.stdin)
print obj

The output should look like so: 

$ jq '.geometry' geoJson.json | python myPythonProgram.py
{u'crs': {u'type': u'name', u'properties': {u'name': u'foo'}}, u'type': u'Polygon', u'coordinates': [[[5, 2], [5.4, 3], [3, 2.1]]]}

Upvotes: 1

Related Questions