CODEWITHSUNDEEP
node.jspuppeteergoogle-chrome-headlessserverside-javascript
web_master
web_master

Reputation: 31

How to Configure Puppeteer to Properly Render External JS Pages? Works for Localhost URLs only

I am trying to configure server-side rendering for external Javascript Pages. I am using Puppeteer for this purpose and when I provide any external URL (not localhost pages), Puppeteer fetches only the source code of the URL (what you may see in view-source mode of the requested page), without rendering the entire DOM. When I try to SSR any localhost Javascript page URL (a page generated by the same node js server on my local host) - all works fine.

Please advise whether I am missing something or I need to try another approach.

I managed to configure puppeteer with all dependencies on my localhost as below:

Currently the 'html' variable returns only the source code of the fetched URL, I need to receive fully rendered DOM of the requested URL.

Code in server.js

var puppeteer = require('puppeteer');

async function ssr(url) {
  console.info('rendering the page in ssr mode');
  const browser = await puppeteer.launch({headless: true});
  const page = await browser.newPage();

  try {
    await page.goto(url, {waitUntil: 'domcontentloaded'});
  } catch (err) {
    console.error(err);
    throw new Error('page.goto/waitForSelector timed out.');
  }

  const html = await page.content();
  await browser.close();
  return {html};
}

module.exports = ssr;

Code in app.js

var err = require('http-errors');
var express = require('express');
var path = require('path');
var ssr = require('./ssr.js');

var app = express();

app.listen(3000, function(){ console.info('Server listening on port '); });

app.use('/index/', async(req, res) => {
  const { html } = await ssr(`www.example.com`);
  return res.send(html);
});

Upvotes: 3

Views: 4320

Answers (2)

Thomas Dondorf
Thomas Dondorf

Reputation: 25240

The problem is likely, that you are not giving the page enough time to render the DOM contents. {waitUntil: 'domcontentloaded'} will only wait for the DOMContentLoaded event, not for any AJAX requests or DOM modifications.

Try to use 'networkidle0' or 'load' as waitUntil value of the page.goto function instead.

If that does not work, you have two options:

Upvotes: 4

Rakan Habab
Rakan Habab

Reputation: 146

try 

const html = 'what ever you selector is but since you want the html just type in "html"'; 

let gg = await page.evaluate((sel) => {
        let element = document.querySelector(sel);
        console.log ('got Boom');
        return element? element.innerHTML: null;
    }, html);


    console.log (gg);

Upvotes: 0

Related Questions