Reputation: 4097
Is there a node
/ npm
module which would scrape the contents of a web page or a given URL and execute and evaluate it's JavaScript as a real-life web browser, including XMLHttpRequest
object and alike?
I thought that scraperjs was the solution, but sadly it isn't.
All help very appreciated!
Upvotes: 3
Views: 7953
Reputation: 4675
Headless browsers based on rendering engine:
Puppeteer is the official automation tool for Chrome. It only supports Chrome/Chromium browsers, (with experimental support for Firefox).
Playwright is an alternative to Puppeteer, with support for more browsers, written by the original creators of puppeteer, who jumped ship from Google to Microsoft. It supports Chrome, Firefox and Safari browsers
Previous versions of my answer, included PhantomJS, SlimerJS,CasperJS and Nightmare all of which are no longer maintaned. For modern browsers, just use one of the above
Upvotes: 7