CODEWITHSUNDEEP
pythonurllib2
Avi
Avi

Reputation: 366

Fetch html content from a destination url that is on onload of the first site in urllib2

I am trying to fetch the HTML content of a website using urllib2. The site has a body onload event that submit a form on this site and hence it goes to a destination site and render the details I need.

response = urllib2.urlopen('www.xyz.com?var=999-999')

www.xyz.com contains a form that is posted to "www.abc.com", this action value varies depending upon the content in url 'var=999-999' which means action value will change if the var value changes to '888-888'

response.read()

this still gives me the html content of "www.xyz.com" , but I want that of resulting action url. Any suggestions of fetching the html content from the final page?

Thanks in advance

Upvotes: 0

Views: 302

Answers (1)

Nelson Benítez León
Nelson Benítez León

Reputation: 50603

You have to figured out the call to that second page, including parameters sent, so you can make that call yourself from your python code, best way is navigate first page with google chrome page inspector opened, then go to Network tab where the POST call would be captured and you can see the parameters sent and all. Then just recreate that same POST call from urllib2.

Upvotes: 1

Related Questions