CODEWITHSUNDEEP
node.jspuppeteer
gbestard
gbestard

Reputation: 1177

Check if strings exists with Puppeteer

I'm trying to use Puppeteer to find if an array of strings exist on a webpage. I'm using this code:

runner.exec(
    'php -r \'$translations = include("translations.php"); print json_encode($translations);\'',
    function (err, stdout, stderr) {
        var translations = JSON.parse(stdout);
        var values = Object.values(translations);
        const puppeteer = require('puppeteer');

        (async () => {
            const browser = await puppeteer.launch();
            const page = await browser.newPage();
            await page.goto('https://example.com');
            values.forEach(element => {
                try {
                    page.$$eval('body', (elements) =>
                        elements.some((el) => el.textContent.includes(element))
                    );
                } catch (error) {
                    console.log(error);
                }
            })
            await browser.close();
        })();
    }
);

But all I get is

(node:6701) UnhandledPromiseRejectionWarning: Error: Protocol error (Runtime.evaluate): Target closed.
    at /var/www/v4/tests/utils/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:208:63
    at new Promise (<anonymous>)
    at CDPSession.send (/var/www/v4/tests/utils/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:207:16)
    at ExecutionContext._evaluateInternal (/var/www/v4/tests/utils/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:162:18)
    at ExecutionContext.evaluateHandle (/var/www/v4/tests/utils/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:151:21)
    at /var/www/v4/tests/utils/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:101:44
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
(node:6701) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict`

Does anyone knows how can I make it work? Thanks

Upvotes: 0

Views: 1472

Answers (2)

Sezerc
Sezerc

Reputation: 179

It looks like you are closing the browser before your values.forEach method finishes running. Making it Promise and waiting for it to resolve might solve the problem.

Upvotes: 1

vsemozhebuty
vsemozhebuty

Reputation: 13782

  1. You need to await promises (for...of loop would be better for this).
  2. All checks in a one .evaluate() would be more fast.
  3. innerText can be more readable than textContent.

You can try something like this:

const puppeteer = require('puppeteer');

const values = ['domain', 'examples', 'documents', 'foo bar'];

(async () => {
    const browser = await puppeteer.launch();
    const [page] = await browser.pages();
    await page.goto('https://example.com');

    const matches = await page.evaluate((strings) => {
      const text = document.body.innerText;
      return strings.filter(string => text.includes(string));
    }, values);

    console.log(matches);
    await browser.close();
})();

Upvotes: 0

Related Questions