CODEWITHSUNDEEP
pythonencodingutf-8python-requests
raecer
raecer

Reputation: 195

Requests POST encoding with python

I'm posting this character "ç" that is usually sent through a form, to a website I don't control. I'm having trouble with the encoding of the data. Trying to send data without encoding it to utf-8 produces this error (correction: this error was for another special character, mistake!):

File "D:\Python34\lib\http\client.py", line 1181, in _send_request
body = body.encode('iso-8859-1')
UnicodeEncodeError: 'latin-1' codec can't encode character '\u0192' in position 512: ordinal not in range(256)

I'm reading the data from a .csv file saved with utf-8 encoding (is this part of the problem?)

After using .encode('utf-8') and a bit of trial and error:

Posting this: ç Displays on their site as: ç

Posting this: ç Displays on their site as: ç

Sample of shortened code:

headers = {
'Accept' : 'application/json, text/javascript, */*; q=0.01',
'Content-Type' : 'application/x-www-form-urlencoded; charset=UTF-8'
}

data = {
"info" : "ç"
}

r = requests.post('www.urlgoeshere.com', headers=headers, data=json.dumps(data,ensure_ascii=False).encode('utf-8'))

Anyone got any tips on how I can send these special characters and have them display correctly? Or should I just give up and edit them all out. Their website form manages it anyway so I would like to as well.

Edit for more complete .csv code:

with open('data.csv', newline='') as dataFile:
dataReader = csv.reader(dataFile)
for row in dataReader:
    data = {
    "values":{
    "title_id":row[1],
    "title":row[0],
    "other stuff":[{"foo":"bar",
    "too":"foobar"}]}
    }

It's in that loop that i do the request as well. Data.csv is a comma-delimited file saved as utf-8.

Upvotes: 1

Views: 6464

Answers (1)

raecer
raecer

Reputation: 195

The simplest answer to this question was that i was opening the datafile without specifying the encoding. Had i added the encoding="utf-8" to the open function, and encoded the request as utf-8 this would have been solved rather quickly.

Upvotes: 1

Related Questions