CODEWITHSUNDEEP
pythonseleniumselenium-webdriver
swor
swor

Reputation: 901

Selenium proxy with authentication

I have to use selenium and proxy with authentication. I have a few constraints

  1. I can't use selenium-wire (only pure selenium allowed)
  2. I have to use headless mode (e.g. chrome_options.add_argument("--headless"))

I read this answer Python proxy authentication through Selenium chromedriver but it doesn't work for headless mode.

Is it possible to use proxy with authentication in my case? (browser(Chrome, Firefox) is not important)

I need python function to create selenium webdriver object authenticate to proxy

Upvotes: 5

Views: 1989

Answers (2)

Joseph Bolton
Joseph Bolton

Reputation: 34

It's the future, and there is a simpler solution now! (I'm using Chrome here)

pip install seleniumbase # tested on seleniumbase==4.30.8

import re
from types import SimpleNamespace

from seleniumbase import Driver

proxy_args = SimpleNamespace(
    username="your-proxy-server-username",
    password="your-proxy-server-password",
    port=6969,
    endpoint="your.proxy.server.domain",  # e.g. bright data's is "brd.superproxy.io",
)
proxy_args.auth_url = f"{proxy_args.username}:{proxy_args.password}@{proxy_args.endpoint}:{proxy_args.port}"

for _ in range(3):
    selenium_driver = Driver(uc=True, headless2=True, proxy=proxy_args.auth_url)
    selenium_driver.uc_open_with_reconnect("https://ipinfo.io/ip", 3)
    print(
        "your IP address is ",
        re.search(r"(\d+\.){3}\d+", selenium_driver.page_source).group(),
    )
    selenium_driver.close()

Upvotes: 0

CatChMeIfUCan
CatChMeIfUCan

Reputation: 569

you can't because you need a GUI to handle it with selenium in your case so I would recommend using a virtual display like Xvfb display server

You can use PyVirtualDisplay (a Python wrapper for Xvfb) to run headless.

for Linux

sudo apt-get install firefox xvfb

install virtual display for python

pip install pyvirtualdisplay

then

from pyvirtualdisplay import Display
from selenium import webdriver

display = Display(visible=0, size=(800, 600))
display.start()

# now Firefox will run in a virtual display. 
# you will not see the browser.
browser = webdriver.Firefox()
browser.get('http://www.google.com')
print browser.title
browser.quit()

display.stop()

in this case, you do not need to add options.add_argument("--headless") argument and you can follow the answers commented above as a solution or doing it your way but I think this is the best solution for using pure selenium for proxy

Upvotes: 2

Related Questions