CODEWITHSUNDEEP
pythonpython-3.xweb-scrapingpython-requests
ppxx
ppxx

Reputation: 198

How can I use requests.post to submit a form on a website?

I'm trying to submit the query form on every ad posted on https://immoweb.be using requests. Whenever the request is sent a unique x-xsrf-token is generated and attached with the Header in POST request. So, after every few hours, 419 error occurs due to token expiration. I'm using https://curl.trillworks.com/ for creating the header and payload for the python script.

import requests
header1 = {
'authority': 'www.immoweb.be',
'accept': 'application/json, text/plain, */*',
'x-xsrf-token': 'value here',
'x-requested-with': 'XMLHttpRequest',
'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.128 Safari/537.36',
'content-type': 'application/json;charset=UTF-8',
'origin': 'https://www.immoweb.be',
'sec-fetch-site': 'same-origin',
'sec-fetch-mode': 'cors',
'sec-fetch-dest': 'empty',
'referer': 'https://www.immoweb.be/en/classified/apartment/for-sale/merksem/2170/9284079?searchId=607a09e4e4532',
'accept-language': 'en-US,en;q=0.9',
'cookie': 'Cookie data here',
}
data1 = '{"firstName":"Name here","lastName":"Name here","email":"[email protected]","classifiedId":9284079,"customerIds":\[add ehre\],"phone":"","message":"I am interested in your property. Can you give me some more information in order to plan a possible visit? Thank you.","isUnloggedUserInfoRemembered":false,"sendMeACopy":false}'
response = s.post('https://www.immoweb.be/nl/email/request', headers=header1, data=data1)  
print (response.status_code)  
print (response.cookies)

Upvotes: 1

Views: 102

Answers (1)

Andrej Kesely
Andrej Kesely

Reputation: 195418

You can get new X-XSRF-TOKEN from the cookie, for example:

import requests
from urllib.parse import unquote

url = "https://www.immoweb.be"

headers = {
    "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:87.0) Gecko/20100101 Firefox/87.0"
}

json_data = {
    "classifiedId": 9234221,  # <-- change this
    "customerIds": [305309],  # <-- change this
    "email": "[email protected]",
    "firstName": "xxx",
    "isUnloggedUserInfoRemembered": False,
    "lastName": "xxx",
    "message": "I am interested in your property. Can you give me some more information in order to plan a possible visit? Thank you.",
    "phone": "",
    "sendMeACopy": False,
}

with requests.Session() as s:
    # this is only for getting first "XSRF-TOKEN"
    s.get(url)
    token = unquote(s.cookies["XSRF-TOKEN"])

    # use this token to post request:
    headers["X-Requested-With"] = "XMLHttpRequest"
    headers["X-XSRF-TOKEN"] = token
    status = s.post(
        "https://www.immoweb.be/en/email/request",
        headers=headers,
        json=json_data,
    ).json()

    print(status)

    # new XSRF-TOKEN is here:
    token = unquote(s.cookies["XSRF-TOKEN"])
    headers["X-XSRF-TOKEN"] = token

    # repeat
    # ...

Prints:

{'success': True}

Upvotes: 1

Related Questions