Reputation: 13
All the articles about web scraping in Python I came across quoted BeautifulSoup and Selenium as the main tools to use to retrieve HTML and JavaScript content from a webpage.
However, I have not found a way to automatically update the data received from this same webpage without the need to send it a new request.
Is it impossible to do in Python what you would do with Google Chrome, that is, simply opening a webpage and looking at it being automatically updated by the host itself?
Upvotes: 1
Views: 522
Reputation: 514
Yes, it is possible. For example, you can download elements of the DOM tree in a loop if the content is dynamically modified using JavaScript. You can also call your own JavaScript to perform a specific action. Alternatively, you can even write and upload your own browser plugin. In general, in selenium you can control the browser to a large extent, just like a normal user, without the need to refresh the page again.
Upvotes: 1