log-in to a website, and use its webpages - Python

Question

I am need to log-in to a website, then surf it's pages and look for the page with the information i need (let's call it a ball), this ball is created dynamically (i mean that the ball, is sometimes can be presented on one page, an sometimes on another and sometimes not at all) as log as i am connected to the website with my e-mail & password.

My goal is, to be able to use a python (that is what i know so far) script, to log-into the website and then load it's webpages, and simply search for the ball and prints me the page it has found it.

I am currently stuck now on the first step, which is how to log into the web site with the python script, as far as i can tell the website in question is using a regular html form field, and in the input filed there is an onclick=someJavaScript() event.

To sum up my question:

• What is the required python module\library that i should use in order to log-into the website ?
• How can i retrieve the website's pages while i am logged-in ?

BTW, I have done a search about the ability to log-into a website through a python script, and i have got many different answers, but since here i am dialing with a JavaScript and the onclick event, I couldn't find an answer on how to do it .

I am using Python 3.x

Answer 1

The mechanize library is made to do these kinds of scripting easier. It will not handle Javascript, so you will still have to figure out what the javascript does through browser development tools like Firebug.

You might also try to disable javascript in the browser and see if you still can use the site.