CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsouppython-requests
user10445685
user10445685

Reputation:

How to handle html form in python?

I have a html page to parse and form in it looking like that:

<select id="limit" name="limit" class="inputbox input-mini" size="1" onchange="this.form.submit()">
    <option value="5">5</option>
    <option value="10" selected="selected">10</option>
    <option value="15">15</option>
    <option value="20">20</option>
    <option value="25">25</option>
    <option value="30">30</option>
    <option value="50">50</option>
    <option value="100">100</option>
    <option value="0">All</option>
</select>

This form affects on how page looks(limit of displayed items). How can I change an option to "All" and submit this form to get all elements?

Upvotes: 0

Views: 509

Answers (1)

Maurice
Maurice

Reputation: 13197

As described in my comment I'd advise you to research, how HTML forms and inputs work.

The URL you need to send your data to is either determined by the action parameter or encoded in some Javascript function. The HTTP method you need to use is set in the method parameter of the form or get by default.

Here's how you could use that in python:

Adapted from the documentation of the requests module.

import requests

# You might need to choose the get method here depending on the value of the forms method parameter
response = requests.post("your-url.here", data={'limit': 0})

print(response.text)

Edit

In response to the Comments I'll add the example for get Requests as well

import requests

response = requests.get("your-url.here", params={'limit': 0})

print(response.text)

The version has the advantage compared to the comment, that it will take care of URL-Encoding special characters for you.

Upvotes: 2

Related Questions