CODEWITHSUNDEEP
javascriptpythonweb-scrapingmechanicalsoup
Stiven Diaz
Stiven Diaz

Reputation: 23

Mechanicalsoup whit interactive pages (without open browser)

I have a problem with interactives pages with forms, because I have to select a element from the form and later render the current page and to scrape it

The form is a simple radio buttons, it contains an onclick action and call a js function that write in a DOM element

<input id="RBLOpcionBuscar_2" type="radio" name="RBLOpcionBuscar" value="Todas" onclick="javascript:setTimeout('__doPostBack(\'RBLOpcionBuscar$2\',\'\')', 0)" />

I find diferents options to get elements generated by javascript: dryscrape, requests_html, python-qt4 and selenium but I do not know how to integrate them and I do not want to use Selenium because it opens a browser

Respects!

Upvotes: 1

Views: 519

Answers (1)

Daniel Amariles
Daniel Amariles

Reputation: 11

You may want to select your option

browser["RBLOpcionBuscar"] = "Todas"

And then send the hidden inputs

 # option of query calls
 browser.get_current_form().set("__EVENTTARGET", "GVConvocatorias", True)
 # call identifier
 browser.get_current_form().set("__EVENTARGUMENT", "Page$" + str(jump_page), True)

here it is an example

https://github.com/danipilze/BECAS_ICETEX/blob/master/main.py

Upvotes: 1

Related Questions