CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
Kiran
Kiran

Reputation: 8538

Screen Scraping a Javascript based webpage in Python

I am working on a screen scraping tool in Python. But, as I look through the source of the webpage, I noticed that most of the data is coming through Javascript.

Any idea, how to scrape javascript based webpage ? Any tool in Python ?

Thanks

Upvotes: 3

Views: 6709

Answers (3)

hoju
hoju

Reputation: 29472

I use webkit, which is the browser renderer behind Chrome and Safari. There are Python bindings to webkit through Qt.

And here is a full Python example to execute JavaScript and extract the final HTML.

Upvotes: 4

Will
Will

Reputation: 4528

You can use the QtWebKit module of the PyQt4 library

Upvotes: 3

unutbu
unutbu

Reputation: 880697

Scraping javascript-based webpages is possible with selenium. In particular, try the Selenium WebDriver.

Upvotes: 5

Related Questions