CODEWITHSUNDEEP
javascriptnode.jsheadless-browserpuppeteer
Ashwin Nair
Ashwin Nair

Reputation: 35

how do I set a proxy server in my headless chrome request using Node JS Puppeteer

/I am running a headless search request on chrome and i need to access a proxy server/ 

const puppeteer = require('puppeteer');
var url="https://www.google.com/search?q=";
var keyword="hotels";
var urls;
var desktopUserAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36";
const response=[];
var i=0;
var userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36";

(async () => {

  const browser = await puppeteer.launch({headless: false });
  const page = await browser.newPage();
  urls=url+keyword;

page.setUserAgent(userAgent);
  response[i]=await page.goto(urls);
  console.log(await browser.version());
})();

//i need to able to access a proxy server in order to google search

Upvotes: 3

Views: 10559

Answers (2)

Cuadrix
Cuadrix

Reputation: 533

I made a module that does this. It's called puppeteer-page-proxy. It supports setting a proxy for an entire page, or if you like, it can set a different proxy for each request.

First install it:

npm i puppeteer-page-proxy

Then require it:

const useProxy = require('puppeteer-page-proxy');

Using it is easy; Set proxy for an entire page:

await useProxy(page, 'http://127.0.0.1:8000');

If you want a different proxy for each request,then you can simply do this:

await page.setRequestInterception(true);
page.on('request', req => {
    useProxy(req, 'socks5://127.0.0.1:9000');
});

Then if you want to be sure that your page's IP has changed, you can look it up;

const data = await useProxy.lookup(page);
console.log(data.ip);

It supports http, https, socks4 and socks5 proxies, and it also supports authentication if that is needed:

const proxy = 'http://login:[email protected]:8000'

Repository: https://github.com/Cuadrix/puppeteer-page-proxy

Upvotes: 2

Md. Abu Taher
Md. Abu Taher

Reputation: 18846

You can pass in a proxy like this in an argument,

const options = {
        headless: false,
        args: [
            `--proxy-server=${proxyIP:proxyPORT}`,
            `--ignore-certificate-errors`
        ]
};
const browser = await puppeteer.launch(options);

If you want proxy authentication, you can use the following in your page object,

await page.authenticate(user, pass);

Upvotes: 4

Related Questions