CODEWITHSUNDEEP
pythonselenium
Viktor
Viktor

Reputation: 1046

How to dynamically change proxies in chrome driver

I have a problem changing proxies, every time i load new page, it creates another browser process. I found solution for Firefox, but none for Chrome browser.

self.options = webdriver.ChromeOptions()
self.options.add_argument("--start-maximized")
self.options.add_argument("--disable-popup-blocking")
self.options.add_argument('--proxy-server=%s' % 'proxy')

Firefox solution: Python Selenium Webdriver - Changing proxy settings on the fly

Upvotes: 6

Views: 8099

Answers (6)

kaliiiiiiiii
kaliiiiiiiii

Reputation: 1135

That's now possible using Selenium-Profiles

You can install it with pip install selenium-profiles.

It builds a Chrome-Extension which gets remotely controlled over websockets or optionally uses seleniumwire.

Example script:

from selenium_profiles.webdriver import Chrome
from selenium_profiles.profiles import profiles

profile = profiles.Windows() # or .Android()

driver = Chrome(profile=profile, injector_options=True)

driver.profiles.proxy.set_single("http://user2:pass2@example_host.com:41149")

print(driver.profiles.proxy.proxy) # current proxy

driver.quit()  # Execute on the End!

Disclaimer: I am the author Selenium-Profiles

Upvotes: 0

Alexei Dom
Alexei Dom

Reputation: 147

Using Selenium-Wire

I needed undetected chrome, but it's not needed.

>>> import seleniumwire.undetected_chromedriver as uc
>>> chrome_options = uc.ChromeOptions()
>>> driver = uc.Chrome(
...     options=chrome_options,
...     seleniumwire_options={}
... )
>>> driver.proxy = { 'https': 'https://IP:PORT'}
>>> driver.get('https://api.ipify.org?format=json')
>>> driver.proxy = { }
>>> driver.get('https://api.ipify.org?format=json')

It does take about 5 seconds though, but works perfectly.

Upvotes: 3

tenbits
tenbits

Reputation: 8008

Once chrome instance is launched, you can't change the proxy, as the changes to capabilities have no effect. But there is a workaround to the issue. You have to use a local proxy, which will target the parent proxies(your desired proxies). In this way you create the proxy chain. And when you want to change the target proxy, you just have to reconfigure the local one, so it targets the new server. Your chrome instance continue to communicate with the local one.

I'm using the squid Cache

You set your target proxy like this

cache_peer PARENT_IP parent PARENT_PORT 0  no-query default login=USERNAME:PASSWORD connect-fail-limit=99999999  proxy-only
never_direct allow all

in squid's config

You have to script file modification every time, you want to set the new proxy. Afterward point squid to reread the configuration.

$ squid -d 0 -k reconfigure

I've wrote more in depth article to the problem:

https://dev.kit.eco/selenium-webdriver-for-chrome-how-to-change-the-proxy-at-runtime-a-workaround

Upvotes: 1

Samuel Chen
Samuel Chen

Reputation: 2273

After investigated, I found the solution to dynamically change proxy with Chrome (work on selenium 3.141.0). The key point is start_session() method. Each time when you requesting a new url, you could start a new session without starting a new browser.

   proxy = get_new_proxy()     # x.x.x.x:y
   
   c = {
       "proxyType": "MANUAL",
       "httpProxy": proxy,
       "sslProxy": proxy
   }
   
   cap = webdriver.DesiredCapabilities.CHROME.copy()
   cap['proxy'] = c
   driver.start_session(cap)
   try:
       b.get('https://whatismyip.com')
   except Exception as e:
       print(e)

p.s. selenium.webdriver.common.proxy.Proxy.add_to_capabilities() may also be used when specifying proxy (so you do not need to use the c dict above.)

Upvotes: -1

liuxun
liuxun

Reputation: 241

I found a solution in Java/Scala, I find that the ChromeOptions extends to MutableCapabilities, as the name suggests, is mutable, so I get the following code in scala, Java needs to change a little

val proxy = new org.openqa.selenium.Proxy()
val proxyStr = "127.0.0.1:1080"
proxy.setHttpProxy(proxyStr)
val option = new ChromeOptions()
option.setProxy(proxy)
chromeDriver.getCapabilities.merge(option) // will change proxy used by the driver

Upvotes: 3

C. Peck
C. Peck

Reputation: 3717

Try something like

ChromeOptions options = new ChromeOptions();
proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.HttpProxy =
proxy.SslProxy = "127.0.0.1:3330";
options.Proxy = proxy;
options.AddArgument("ignore-certificate-errors");
var chromedriver = new ChromeDriver(options);

Upvotes: -1

Related Questions