CODEWITHSUNDEEP
pythonpython-3.xweb-scraping
sam pol
sam pol

Reputation: 1

Automate download in web using python

I have a script that will login to a site, and then it will click a link and then it needs to export the data to excel file.

The script already have a login to the site It is lacking on the download part

with requests.Session() as s:
  url="https://website"
  r=s.get(url,headers=headers)
  soup=BeautifulSoup(r.text,"lxml")
  login_data['grpstub']=soup.find('input' , attrs={'name':'grpstub'})['value']
  login_data['__VIEWSTATE']=soup.find('input' , attrs={'name':'__VIEWSTATE'})['value']
  login_data['__VIEWSTATEGENERATOR']=soup.find('input' , attrs={'name':'__VIEWSTATEGENERATOR'})['value']
  login_data['__VIEWSTATEENCRYPTED']=soup.find('input' , attrs={'name':'__VIEWSTATEENCRYPTED'})['value']
  login_data['__EVENTVALIDATION']=soup.find('input' , attrs={'name':'__EVENTVALIDATION'})['value']
  r = s.post(url,data=login_data,headers=headers)
  print(r.content)

I need to have a script that will do the below export to excel

click to see image

Upvotes: 0

Views: 89

Answers (1)

collerek
collerek

Reputation: 261

BeautifulSoup is used to extracts/parse html and xml.

If you want to automate actions (click buttons/ input forms etc.) on a web page you should use something like Selenium.

That way you can find an element in the page structure (selenium has a lot of builtin selectors) and issue an value (form input) or event (click) etc.

Upvotes: 2

Related Questions