CODEWITHSUNDEEP
pythonjsoncurl
Malluce
Malluce

Reputation: 141

Reading JSON files from curl in Python

Let's say I have a command in Python that looks like

command = 'curl ...etc" > result.json'
subprocess.call(command, shell = True)
file = open("result.json").read()

What it does right now is GET from the place I curl and store the result in result.json, then I open it to read it. I wonder if there's way I can read it directly without storing to local first?

Upvotes: 11

Views: 44049

Answers (3)

Zgpeace
Zgpeace

Reputation: 4477

python3 use curl to request data
For example, change the curl to python in the below.

$ curl -XPOST http://httpbin.org/post -H "Content-Type:application/json" -d '{"attribute":"value"}'

intstall urllib3 using $ python -m pip install urllib3


import urllib3
import json
data = {'attribute': 'value'}
encoded_data = json.dumps(data).encode('utf-8')
r = http.request(
    'POST',
    'http://httpbin.org/post',
    body=encoded_data,
    headers={'Content-Type': 'application/json'}
)
json.loads(r.data.decode('utf-8'))['json']

print result 

{'attribute': 'value'}

ref: https://urllib3.readthedocs.io/en/latest/user-guide.html

Upvotes: 1

Esparta Palma
Esparta Palma

Reputation: 745

You can use the stdlib (json & urllib2) to avoid the use of external commands: 

import json
import urllib2
url = "http://httpbin.org/get"
response = urllib2.urlopen(url)
data = response.read()
values = json.loads(data)

But I'd recommend to use requests to simplify your code. Here a sample from the docs:

import requests
r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
r.status_code
200
r.headers['content-type']
'application/json; charset=utf8'
r.encoding
'utf-8'
r.text
u'{"type":"User"...'
r.json()
{u'private_gists': 419, u'total_private_repos': 77, ...}

Python 3 Update

Please consider that in Python3 urllib2 does not exists anymore, you should use urllib which comes in the standard library

req = urllib.request.Request(url)
response = urllib.request.urlopen(req)
data = response.read()
values = json.loads(data)

Upvotes: 32

jfs
jfs

Reputation: 414875

In general, if you have a command that prints to its stdout then you could get the output without storing it on the disk using subprocess.check_output:

from subprocess import check_output

output = check_output(['source', 'arg1', 'arg2'])

In your case, you could use urllib2 or requests Python modules instead of curl command as shown in @Esparta Palma's answer.

Upvotes: 2

Related Questions