CODEWITHSUNDEEP
javascriptnode.jsconcurrencypuppeteerpuppeteer-cluster
user2912391
user2912391

Reputation: 67

Run multiple puppeteer executions with puppeteer-cluster

How do you run multiple concurrent executions with puppeteer-cluster?

I have 5 as max concurrency, but doing await cluster.execute() only runs 1 at a time.

const cluster = await Cluster.launch({
   monitor: false,
   concurrency: Cluster.CONCURRENCY_BROWSER,
   maxConcurrency: 5,
})

for(let url of urls) {
   await cluster.execute(url)
}

I want all 5 to be executed concurrent. The alternative is await cluster.queue(), but after a while, the memory is being eaten up, because there is no way to check for current queue length/size.

Upvotes: 1

Views: 1189

Answers (1)

Ahmed Belghith
Ahmed Belghith

Reputation: 1

change concurrency: Cluster.CONCURRENCY_BROWSER to concurrency: Cluster.CONCURRENCY_CONTEXT

Upvotes: 0

Related Questions