CODEWITHSUNDEEP
javascriptpythonseleniumbeautifulsoupscreen-scraping
MiniCoder
MiniCoder

Reputation: 13

Web scraping: update data like in Chrome?

All the articles about web scraping in Python I came across quoted BeautifulSoup and Selenium as the main tools to use to retrieve HTML and JavaScript content from a webpage.

However, I have not found a way to automatically update the data received from this same webpage without the need to send it a new request.

Is it impossible to do in Python what you would do with Google Chrome, that is, simply opening a webpage and looking at it being automatically updated by the host itself?

Upvotes: 1

Views: 522

Answers (1)

OceanFire
OceanFire

Reputation: 514

Yes, it is possible. For example, you can download elements of the DOM tree in a loop if the content is dynamically modified using JavaScript. You can also call your own JavaScript to perform a specific action. Alternatively, you can even write and upload your own browser plugin. In general, in selenium you can control the browser to a large extent, just like a normal user, without the need to refresh the page again.

Upvotes: 1

Related Questions