CODEWITHSUNDEEP
pythonhttpcookieshttp-headershttprequest
Roman Rdgz
Roman Rdgz

Reputation: 13264

HTML 501 error using Python Requests

I'm writing a Python script to search for train tickets. So my first step was trying to make a ticket search at https://venta.renfe.com/vol/inicioCompra.do

I have tried mechanize, but it gets:

RuntimeError: maximum recursion depth exceeded while calling a Python object

as soon as I open the url, probably due to some bad format at that website.

So next attempt was using Python Requests library, and use the same request headers I can see at firebug. So my code looks like:

import requests

headers = {'Cache-Control': 'max-age=0',
           'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
           'Accept-Encoding': 'gzip,deflate,sdch',
           'Accept-Language': 'es,en-GB;q=0.8,en;q=0.6',
           'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36',
           'Content-Length': '118',
           'Content-Type': 'application/x-www-form-urlencoded',
           'Connection': 'keep-alive',
           'Host': 'venta.renfe.com'}

cookies = {'target': '_self',
           'pagina': '/vol/index.do',
           'mensajeErrorSesion': 'null (null-U014)',
           'url_logout': '/vol/index.do',
           'tipoUsuario': 'N',
           'JSESSIONID': '0000W1mXLh9qNdE6l-qaEExFc9Y:15df38flv',
           'org.springframework.web.servlet.i18n.CookieLocaleResolver.LOCALE': 'es_ES',
           's_cc': 'true',
           's_fid': '5C053C8CEF00A17B-2B7E62472A63CF66',
           's_nr': '1377790666229-New',
           'gpv_p6': 'Venta%3APagina%20Principal',
           's_sq': 'renfeprod%3D%2526pid%253DVenta%25253APagina%252520Principal%2526pidt%253D1%2526oid%253Djavascript%25253AestacionesAccesibles%252528%252529%25253B%2526ot%253DA Host:venta.renfe.com Origin:https://venta.renfe.com Referer:https://venta.renfe.com/vol/inicioCompra.do'}

payload = {'IdOrigen': 'Madrid (*)',
           'IdDestino': 'Oviedo',
           'FechaIdaSel': '15/11/2013',
           'FechaVueltaSel': '17/11/2013'}

r = requests.post('https://venta.renfe.com/vol/inicioCompra.do', data=payload, cookies=cookies, headers=headers)
print r

But then I'm receiving 501 HTML code error: Not implemented. I guess I'm missing something, probably cookie information, but I don't know where to get that cookie info, or if I'm missing anything else.

Any idea?

Upvotes: 1

Views: 3481

Answers (1)

Prahalad Deshpande
Prahalad Deshpande

Reputation: 4767

I looked through the site and it appears that you are using a GET HTTP method to retrieve the data when what you actually need is a POST.

Typically an HTTP 501 is sent across as a response to the client, when the web server does not understand the HTTP verb sent across by the client within the request.

Try changing the code:

r = requests.get('https://venta.renfe.com/vol/inicioCompra.do', data=payload, cookies=cookies, headers=headers)

to something like

r = requests.post('https://venta.renfe.com/vol/inicioCompra.do', data=payload, cookies=cookies, headers=headers)

Note : I have not used Requests, hence you may want to double check the function call parameters. For a quick reference see this link.

Hope this helps - and here is a dump of my header as visible in Chrome. Observe that you are missing the Content-Type, Content-Length parameters within the header. Also note the contents of the cookie. 

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 
Accept-Encoding:gzip,deflate,sdch 
Accept-Language:en-US,en;q=0.8 
Cache-Control:max-age=0 
Connection:keep-alive Content-Length:118 
Content-Type:application/x-www-form-urlencoded 

Cookie:target=_self; pagina=/vol/index.do; mensajeErrorSesion=null (null-U014); url_logout=/vol/index.do; tipoUsuario=N; JSESSIONID=0000W1mXLh9qNdE6l-qaEExFc9Y:15df38flv; org.springframework.web.servlet.i18n.CookieLocaleResolver.LOCALE=es_ES; s_cc=true; s_fid=5C053C8CEF00A17B-2B7E62472A63CF66; s_nr=1377790666229-New; gpv_p6=Venta%3APagina%20Principal; s_sq=renfeprod%3D%2526pid%253DVenta%25253APagina%252520Principal%2526pidt%253D1%2526oid%253Djavascript%25253AestacionesAccesibles%252528%252529%25253B%2526ot%253DA Host:venta.renfe.com Origin:https://venta.renfe.com Referer:https://venta.renfe.com/vol/inicioCompra.do 
    User-Agent:Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36

Upvotes: 1

Related Questions