CODEWITHSUNDEEP
pythonpython-requests
Tennyx
Tennyx

Reputation: 153

Using 'Requests' python module for POST request, receiving response as if it were GET

So I am trying to make a script that checks a reservation availability of a bus. The starting link for this is https://reservation.pc.gc.ca/.

In the reserve box the following needs to be selected:

When those options are entered, it takes you to the following page: https://reservation.pc.gc.ca/Yoho-LakeO'Hara?Calendar

It is my understanding that if I send a POST request to that second link with the correct data it should return the page I'm looking for

If I look in the dev tools network info when I select the correct parameters the form data is:

__EVENTTARGET:

__EVENTARGUMENT: __VIEWSTATE: -reallllly long string-

__VIEWSTATEGENERATOR: 8D0E13E6

ctl00$MainContentPlaceHolder$rdbListReservationType: Events

ddlLocations: 213a1bc9-9218-4e98-9a7f-0f209008e437**

ddlArrivalMonth: 2017-06-16

ddlArrivalDay: 19

ddlNights: 1

ddlDepartureMonth:

ddlDepartureDay:

ddlEquipment:

ddlEquipmentSub:

ddlPartySize:2

ctl00$MainContentPlaceHolder$chkExcludeAccessible: on

ctl00$MainContentPlaceHolder$imageButtonCalendar.x: 64

ctl00$MainContentPlaceHolder$imageButtonCalendar.y: 56

So the code I wrote is:

import requests

payload = {
        '__EVENTTARGET': '',
        '__EVENTARGUMENT': '',
        '__VIEWSTATE':-reallly long string-,
        '__VIEWSTATEGENERATOR': '8D0E13E6',
        'ctl00$MainContentPlaceHolder$rdbListReservationType': 'Events',
        'ddlLocations': '213a1bc9-9218-4e98-9a7f-0f209008e437',
        'ddlArrivalMonth': 2017-06-16,
        'ddlArrivalDay': 19,
        'ddlNights': 1,
        'ddlDepartureMonth': '',
        'ddlDepartureDay': '',
        'ddlEquipment': '',
        'ddlEquipmentSub': '',
        'ddlPartySize': 2,
        'ctl00$MainContentPlaceHolder$chkExcludeAccessible': 'on',
        'ctl00$MainContentPlaceHolder$imageButtonCalendar.x': 64,
        'ctl00$MainContentPlaceHolder$imageButtonCalendar.y': 56
        }

r = requests.get(r"https://reservation.pc.gc.ca/Yoho-LakeO'Hara?Calendar", data=payload)

print r.text

r.text ends up just being the second link as if no parameters were entered - as if I just sent a normal GET request to the link. I tried turning the payload values that are integers into strings, I tried removing the empty key:value pairs. No luck. Trying to figure out what I'm missing.

Upvotes: 4

Views: 817

Answers (2)

sukotto1
sukotto1

Reputation: 190

It looks to me like there are 2 things going on:

  1. @errata was correct, and this should be a POST request. You're about halfway there.

  2. What I noticed though is that it seems to post the form data to Home.aspx and the URL that you see after submitting the form is the result of that processing and subsequent redirect.

You might try posting the form data as json to ./Home.aspx.

I found through Postman that this nearly worked, but I had to specify the content-type in order to get the proper results.

If you need to know how to add header and body instructions to the .post() method, it looks like there is a good example (though perhaps slightly outdated) here: adding header to python request module

Also, fwiw, check out Postman. If you're both inexperienced with requests and with doing it in Python, at least this may lesson some of the burden of trial and error.

Upvotes: 2

errata
errata

Reputation: 6041

You are using

r = requests.get(r"https://reservation.pc.gc.ca/Yoho-LakeO'Hara?Calendar", data=payload)

instead of

r = requests.post(r"https://reservation.pc.gc.ca/Yoho-LakeO'Hara?Calendar", data=payload)

Digging a bit deeper in your problem, I found out that the URL you are calling is actually redirecting to a different URL (returning HTTP response 302):

$ curl -I "https://reservation.pc.gc.ca/Yoho-LakeO'Hara"
HTTP/1.1 302 Found
Cache-Control: private
Content-Length: 77273
Content-Type: text/html; charset=utf-8
Location: https://reservation-pc.fjgc-gccf.gc.ca/GccfLanguage.aspx?lang=eng&ret=https%3a%2f%2freservation.pc.gc.ca%3a443%2fYoho-LakeO%27Hara
Server: Microsoft-IIS/8.0
Set-Cookie: ASP.NET_SessionId=qw4p4e2zxjxx0c2zyq014p45; path=/; secure; HttpOnly
Set-Cookie: CookieLocaleName=en-CA; path=/; secure; HttpOnly
X-Powered-By: ASP.NET
X-Frame-Options: SAMEORIGIN
Date: Wed, 17 May 2017 14:22:53 GMT

However, following the Location from response results also in 302:

$ curl -I "https://reservation-pc.fjgc-gccf.gc.ca/GccfLanguage.aspx?lang=eng&ret=https%3a%2f%2freservation.pc.gc.ca%3a443%2fYoho-LakeO%27Hara"
HTTP/1.1 302 Found
Cache-Control: private
Content-Length: 179
Content-Type: text/html; charset=utf-8
Location: https://reservation.pc.gc.ca:443/Yoho-LakeO'Hara?gccf=true
Server: Microsoft-IIS/8.0
Set-Cookie: ASP.NET_SessionId=rbcuvexfg4fb340ixtcjd1qy; path=/; secure; HttpOnly
Set-Cookie: _gc_lang=eng; domain=.fjgc-gccf.gc.ca; path=/; secure; HttpOnly
X-Powered-By: ASP.NET
X-Frame-Options: SAMEORIGIN
Date: Wed, 17 May 2017 14:24:55 GMT

All this probably results in Requests transforming your POST into GET in the end...

Upvotes: 1

Related Questions