CODEWITHSUNDEEP
pythonweb-scrapingpython-requests
Natasha
Natasha

Reputation: 1521

Search on a webpage using python requests

I want to fetch the html contents of a webpage. I am not sure how to define the search field, I tried the following.

from fake_useragent import UserAgent
import requests


ua = UserAgent()
print(ua.chrome)
header = {'User-Agent': str(ua.chrome)}
print(header)
body = {'Search': '1.1.1.1'}
url = "https://randr.nist.gov/enzyme/Default.aspx"
htmlContent = requests.get(url, data=body) 
print(htmlContent.text)

Could someone suggest how to define the right search field?

Upvotes: 0

Views: 79

Answers (1)

Kostas Charitidis
Kostas Charitidis

Reputation: 3103

Can be easily done with selenium like this:

from selenium import webdriver

search_input = '1.1.1.1'

driver = webdriver.Chrome('chromedriver.exe')
driver.get('https://randr.nist.gov/enzyme/Default.aspx')
driver.find_element_by_id('MainBody_txtSrchAutoFill').send_keys(search_input)
driver.find_element_by_id('MainBody_ImgSrch').click()
result_table = driver.find_element_by_id('MainBody_gvSearch')
print(result_table.text)

Upvotes: 1

Related Questions