CODEWITHSUNDEEP
node.jspuppeteerwebautomation
kazaff
kazaff

Reputation: 177

How to detect the request come from Puppeteer?

I wonder whether or not exists some flag or tag that the website can use it to detect the request came from Puppeteer?

When I ran my code based on Puppeteer to visit the target website, I found that the website seems to know the request was made by Puppeteer.

How can it do?

Upvotes: 6

Views: 6572

Answers (2)

kazaff
kazaff

Reputation: 177

I found a way to cross the limitation. It's an easy way:

const browser = await puppeteer.launch({headless: false, ignoreDefaultArgs: ["--enable-automation"],});

This will let the browser to not setup navigator.webdriver variable.

Upvotes: 5

Nicolò Gasparini
Nicolò Gasparini

Reputation: 2396

If you are running the puppeteer and would like to pass some information to the website to catch your crawling, the best way to do so would be to set a custom user agent:

const browser = await puppeteer.launch({
    args: ['--user-agent=hhh'],
});
const page = await browser.newPage();

See here more info

Viceversa, if you own a website and would like to know if the visits are real or from a bot (puppeteer, a scraper, or anything else) see this answer for some of them. Also this answer

Upvotes: 4

Related Questions