CODEWITHSUNDEEP
phpjqueryweb-scrapingheadless
Ibrahem Al-Badareen
Ibrahem Al-Badareen

Reputation: 91

How to get the HTML code for a web page after it is fully loaded in PHP?

I am trying to get the HTML code for any web page, only and only after it is fully loaded.

I tried CURL, and file_get_contents, and I understand by now that they do not wait for JavaScript to finish.

I know by now that the solution is to use a headless browser. I tried PhantomJS, but it is a little bit inefficient since the only method found to make it wait, is to set a constant timeout period.

Also, I found out that in general, it is almost impossible to get when the page actually fully load, and the best approach was to keep checking for the network data until they completely stop.

I believe to keep checking for the existence of a content in the page, would work just fine for my use, but as fast as I know, the only method to implement that is to use puppeteer package which only works well with NodeJS not PHP.

So, do you guys know about any efficient method to get the HTML code after the page is fully loaded in PHP, without going through the complex process of integrating other programming languages, or other platforms?

Upvotes: 1

Views: 598

Answers (2)

Marek Kus
Marek Kus

Reputation: 150

Maybe something like setting JS on page load finish to get all page content to variable, and then sending that variable to php script via Ajax.

Upvotes: 0

Bill Criswell
Bill Criswell

Reputation: 32921

I don't think you'll be able to accomplish it with PHP since it's not a browser and can't run JavaScript. You can use something like Headless Chromium and do chrome --headless --disable-gpu --dump-dom https://www.chromestatus.com/ which unfortunately can't tell exactly when it's "fully loaded" but you can do it on some type of delay I'm sure.

Upvotes: 1

Related Questions