CODEWITHSUNDEEP
puppeteerpuppeteer-cluster
Tom
Tom

Reputation: 251

Correct way to pass args in puppeteer-cluster via puppeteerOptions

I am trying to use args in my code to use a proxy service I have. If I remove the args altogether things run fine but if I have them in there I get an error stating: Error: Unable to restart chrome. I checked multiple examples and copied the same to my code but it seems to fail. Any ideas on how to implement this correctly?

Code:

const { Cluster } = require('puppeteer-cluster');
const vanillaPuppeteer = require('puppeteer');
const { addExtra } = require('puppeteer-extra');
const Stealth = require('puppeteer-extra-plugin-stealth')


async function main() {
  // Create a custom puppeteer-extra instance using `addExtra`,
  // so we could create additional ones with different plugin config.
  const puppeteer = addExtra(vanillaPuppeteer)
  puppeteer.use(Stealth())

  let proxy_server = 'proxy.soax.com:9000';
  let user = 'some_user_name';
  let pass = 'some_password';

  // Launch cluster with puppeteer-extra
  const cluster = await Cluster.launch({
    puppeteer,
    puppeteerOptions: {
        headless: false,
        args: ['--proxy-server=' + proxy_server,
               '--single-process', 
               '--no-zygote', 
               '--no-sandbox'],
        sameDomainDelay: 1000,
        retryDelay: 3000,
        workerCreationDelay: 3000},
    maxConcurrency: 2,
    concurrency: Cluster.CONCURRENCY_CONTEXT,
    monitor: false,
    skipDuplicateUrls: true
  })

  // Define task handler
  await cluster.task(async ({ page, data: url }) => {
    await page.authenticate({
        username: user,
        password: pass,
    });  
    await page.goto(url)

    const { hostname } = new URL(url)

    console.log(`checking on ${hostname}`)

    await page.screenshot({ path: `${hostname}.png`, fullPage: true })
  })

  // Queue any number of tasks
  cluster.queue('https://whatismyipaddress.com/')

  await cluster.idle()
  await cluster.close()
  console.log(`All done`)
}

main().catch(console.warn)

Upvotes: 1

Views: 2242

Answers (1)

Tom
Tom

Reputation: 251

I played around a bit and discovered by removing the arg --single-process then it works fine.

Upvotes: 2

Related Questions