CODEWITHSUNDEEP
pythonpostcurlurllib2pycurl
Oli
Oli

Reputation: 239810

Posting raw data with Python

I'm playing around with the Google Checkout API and I want to pull it into a Django app. I need to post data to Google using basic http authentication. I've been testing this with curl like this:

curl -d "$(cat mytest.xml)" -u username:password https://url

And that posts the content of my test XML file to Google. And it works fine!

But I'm having problems porting that simple line to Python. I've managed several different ways (httplib2, urllib2, pycurl) of connecting with a password and posting something but the respose is always 400 BAD REQUEST.

Is there a python equivalent for posting block of text to a HTTP Basic auth server? I'm running out of walls to bang my head against.

Apologies for not adding any code. Here are some of my greatest hits. In each, DATA is an XML string. URL, USERNAME and PASSWORD are constant.

req = urllib2.Request(URL)
req.add_header("Authorization", "Basic %s" % base64.encodestring('%s:%s'%(USERNAME, PASSWORD)))
u = urllib2.urlopen(req, DATA)

Gives me a lovely HTTP Error 400: Bad Request

passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, URL, USERNAME, PASSWORD)
authhandler = urllib2.HTTPBasicAuthHandler(passman)
opener = urllib2.build_opener(authhandler)
urllib2.install_opener(opener)
pagehandle = urllib2.urlopen(URL, DATA)

Gives HTTP Error 401: Unauthorized

pycurl.global_init(pycurl.GLOBAL_DEFAULT)
c = pycurl.Curl()
c.setopt(pycurl.URL, URL)
c.setopt(pycurl.USERPWD, "%s:%s" % (USERNAME,PASSWORD))
c.setopt(pycurl.POST, 1)
c.setopt(pycurl.HTTPHEADER, ["Content-type: text/xml"])
c.setopt(pycurl.POSTFIELDS, DATA)
b = StringIO.StringIO()
c.setopt(pycurl.WRITEFUNCTION, b.write)
c.perform()

Seems to struggle with passing the DATA string as a POSTFIELD. I've tried urllib.urlencode()ing DATA in several different ways but 

h = httplib2.Http()
h.add_credentials(USERNAME, PASSWORD)
print = h.request(URL, "POST", body=base64.encodestring(DATA))

The credentials don't seem to do anything - I get an unauthorised message back from Google.

There are more but they're all based on these.

Upvotes: 9

Views: 16373

Answers (3)

JWL
JWL

Reputation: 14201

I've been having similar turmoils with the stdlib packages, until somoneone pointed to the awesome requests that supports basic Http Authentication and other authentication means straight out-of-the-box! And it has a beautiful and simple API it hurts!

requests.post(url, data=DATA, headers=HEADERS_DICT, auth=(username, password))

It supports a host of other necessary features (e.g HTTPS, Digest Authentication, etc) Please check it out if u must...

Upvotes: 12

Andrew Wilkinson
Andrew Wilkinson

Reputation: 10836

Voidspace has an excellent article on using basic auth with urllib2. I've copied the appropriate code snippet below, changed to use POST.

import urllib2

theurl = 'http://www.someserver.com/toplevelurl/somepage.htm'
username = 'johnny'
password = 'XXXXXX'

passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, theurl, username, password)

authhandler = urllib2.HTTPBasicAuthHandler(passman)

opener = urllib2.build_opener(authhandler)

urllib2.install_opener(opener)

pagehandle = urllib2.urlopen(theurl, open("mytext.xml").read())

Without seeing your code it's hard to say why you would be getting a 400 response.

Upvotes: 2

Oli
Oli

Reputation: 239810

While editing my post to include some source, I thought I'd have another crack at httplib2 (mainly because it's comparatively small and pretty compared to the others) and noticed that there's a gaping bug in that its add_credentials(..) method doesn't actually do anything. You can work around this by specifying the header (as I did with urllib2) like this:

resp, content = httplib2.Http().request(URL, "POST", body=DATA, headers={
    "Authorization": "Basic %s" % base64.encodestring('%s:%s' %(USERNAME, PASSWORD)
})

And this works.

Upvotes: 2

Related Questions