CODEWITHSUNDEEP
pythonencodingpython-requests
illright
illright

Reputation: 4043

How to change the form encoding in Python Requests?

I have a Cyrillic string that I'd like to send as form data in Windows-1251 encoding using Python Requests.

Take a look at this example:

import requests
st = 'Искать'
requests.post('http://localhost:8888', data={'test': st})

However, the data of the request becomes this (running nc -l 8888 to view the request):

test=%D0%98%D1%81%D0%BA%D0%B0%D1%82%D1%8C

That is my target string, encoded as UTF-8, which we can easily check:

>>> from urllib.parse import quote
>>> st = 'Искать'
>>> quote(st, encoding='utf-8')  # Matches
'%D0%98%D1%81%D0%BA%D0%B0%D1%82%D1%8C'
>>> quote(st, encoding='cp1251')  # Doesn't match
'%C8%F1%EA%E0%F2%FC'

Now I thought that if I encode the data myself, it would work just fine. 

requests.post('http://localhost:8888', data={'test': quote(st, encoding='cp1251')})

But turns out that this is not the case because while the letters are encoded, the percent signs get encoded again into UTF-8 (%25) and the whole string becomes invalid again

test=%25C8%25F1%25EA%25E0%25F2%25FC

So I'm looking for a way to either disable the built-in Requests' encoding feature or override the encoding value. How can I do it?

I'm using Python 3.5, Requests 2.18.4

Upvotes: 1

Views: 3526

Answers (1)

t.m.adam
t.m.adam

Reputation: 15376

From docs.python-requests:

There are times that you may want to send data that is not form-encoded. If you pass in a string instead of a dict, that data will be posted directly.

So, if you don't want your data to get form-encoded you should use a string. 

st = 'Искать'
data={'test': quote(st, encoding='cp1251')}
data = '&'.join('='.join(i) for i in data.items())
requests.post('http://localhost:8888', data=data)

Upvotes: 2

Related Questions