CODEWITHSUNDEEP
pythonwindowscurlpython-2.7
fusi0n
fusi0n

Reputation: 1079

Curl Is Not Recognized , Using Popen()

p = Popen(["curl", "http://localhost:8983/solr/update/json?commit=true", "--data-binary", "@solrData.json", "-H", "Content-type:application/json"], cwd=r"C:/Users/SOLR/docs", shell=True)

This the code I am using right now, and I get the error that curl is not recognized. When I run the command in the same dir where the solrData.json file is:

curl "http://localhost:8983/solr/update/json?commit=true" --data-binary @solrData.json -H "Content-type:application/json"

it runs perfectly. Curl is in the system path and there everthing works. Also as a comparison this works just fine:

p = Popen("java -jar post.jar solrData.xml", cwd=r"C:/Users/SOLR/docs")

EDIT

    import requests
# open the file to upload
        with open('C:/Users/SOLR/docs/solrData.json', 'rb') as fin:
            # execute the post request
            headers = {'Content-type': 'application/json'}
            r = requests.post("http://localhost:8983/solr/update/json", params=dict(commit="true"), headers=headers, data=fin.read())

This is the solution that works. Thank you zmo and Martijn Pieters for your help.

Upvotes: 1

Views: 2132

Answers (2)

Martijn Pieters
Martijn Pieters

Reputation: 1121844

You are passing in a list of arguments, but set shell=True. Switch off the latter (remove the argument, False is the default) and let Python handle the command:

p = Popen(["curl", "http://localhost:8983/solr/update/json?commit=true", "--data-binary", "@solrData.json", "-H", "Content-type:application/json"], cwd=r"C:/Users/SOLR/docs")

Note that it'll be vastly simpler to install the python-requests library and do the whole task from Python with minimal fuss:

import requests

with open('C:/Users/SOLR/docs/solrData.json', 'r') as solrdata:
    r = requests.post('http://localhost:8983/solr/update/json?commit=true',
        data=solrdata, headers={'Content-type': 'application/json'})

Upvotes: 2

zmo
zmo

Reputation: 24812

from Popen manpage:

On Unix with shell=True, the shell defaults to /bin/sh. If args is a string, the string specifies the command to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This includes, for example, quoting or backslash escaping filenames with spaces in them. If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself. That is to say, Popen does the equivalent of:

Popen(['/bin/sh', '-c', args[0], args[1], ...])

On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.

So you need to issue the following command:

p = Popen(["curl", "http://localhost:8983/solr/update/json?commit=true", "--data-binary", "@solrData.json", "-H", "Content-type:application/json"], cwd=r"C:/Users/SOLR/docs", shell=False)

but why do you want to use subprocess to use curl, whereas you have the pycurl library that makes it easier to use curl? Here's a code inspired by the file upload example:

import os
import pycurl
import cStringIO

# here's a handler to expose the read_callback method
class FileReader:
    def __init__(self, fp):
        self.fp = fp
    def read_callback(self, size):
        return self.fp.read(size)

# here's a buffer to handle the output of the pycurl request
buf = cStringIO.StringIO()

# we open the file
with open('solrData.json', 'rb') as fin:
    # get its size
    filesize = os.path.getsize('solrData.json')

    # initiates curl
    c = pycurl.Curl()

    # setup curl (url, as a post request, file upload size and content, content-type and output buffer)
    c.setopt(c.URL, 'http://localhost:8983/solr/update/json?commit=true')
    c.setopt(pycurl.POST, 1)
    c.setopt(pycurl.POSTFIELDSIZE, filesize)
    c.setopt(pycurl.READFUNCTION, FileReader(f).read_callback)
    c.setopt(pycurl.HTTPHEADER, ["Content-type: application/json"])
    c.setopt(c.WRITEFUNCTION, buf.write)

    # we execute the query
    c.perform()

    # and write the result of the query
    with open('C:/Users/SOLR/docs/result.json') as f:    
        f.write(buf.getvalue())
    # and finally we close the buffer
    buf.close()

or even simpler than using pycurl, you can use the requests library:

import requests

# open the file to upload
with open('solrData.json', 'rb') as fin:
    # execute the post request
    headers = {'Content-type': 'application/json'}
    r = requests.post("http://httpbin.org/get", params=dict(commit=True), headers=headers, data=fin.read())
    # write the result
    with open('C:/Users/SOLR/docs') as f:
        f.write(f.text) # write the json data in text file
        print f.json() # print the json content as python datatypes

HTH

Upvotes: 0

Related Questions