CODEWITHSUNDEEP
pythonweb-scrapingscrapy
Goku Frieza
Goku Frieza

Reputation: 73

Send Request payload using python scrapy

I am trying to scrape a website where I can find the url has a post request with payload parameters as below. I am not sure how to make it to a dictionary in payload and send it in formdata. All the below code comes under Request with payload how do I send this in formdata??

    ------WebKitFormBoundaryj9yKl83Zu7ki71zE
    Content-Disposition: form-data; name="textquery"

    852432-B21
    ------WebKitFormBoundaryj9yKl83Zu7ki71zE--

How to format this and do I send this using python-scrapy?

Upvotes: 1

Views: 1690

Answers (1)

mizhgun
mizhgun

Reputation: 1887

You should send it as raw request body with appropriate header:

body = '''--WebKitFormBoundaryj9yKl83Zu7ki71zE
Content-Disposition: form-data; name="textquery"

852432-B21
--WebKitFormBoundaryj9yKl83Zu7ki71zE--
'''

yield Request(url, method='POST', body=body, 
              headers={'Content-Type': 'multipart/form-data; boundary=WebKitFormBoundaryj9yKl83Zu7ki71zE'})

Upvotes: 1

Related Questions