CODEWITHSUNDEEP
pythonpython-3.xseleniumproxywebdriver
tobloef
tobloef

Reputation: 1891

Python Selenium Webdriver - Changing proxy settings on the fly

I'm currently successfully using the code below to use a proxy with the Selenium webdriver. Unfortunately, I can't seem to make it change the proxy settings without restarting the whole browser. I had hoped that simply updating the proxy settings, just like I did to set the proxy to start with, would change the proxy, but it doesn't seem to work. Any help on this subject would be greatly appreciated.

profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http", proxyAddress)
profile.set_preference("network.proxy.http_port", proxyPort)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile=profile)

Upvotes: 20

Views: 17378

Answers (4)

Nico
Nico

Reputation: 29

Another option without changing the proxy in selenium would be connecting to a rotating proxy-ip. Most proxy providers offer this functionality where a new ip is assigned for each request you send to the same static proxy-ip.

You need to send an actual request to the rotating ip though, so just opening a new page with webdriver.get() won't work, but you can just send a normal get-request via python's request-module and selenium's traffic now will also be routed to the new ip.

Upvotes: 0

Samuel Chen
Samuel Chen

Reputation: 2273

To change proxy on the fly with Chrome (work on selenium 3.141.0, key point is driver.start_session(cap)):

   proxy = get_new_proxy()     # x.x.x.x:y
   
   c = {
       "proxyType": "MANUAL",
       "httpProxy": proxy,
       "sslProxy": proxy
   }
   
   cap = webdriver.DesiredCapabilities.CHROME.copy()
   cap['proxy'] = c
   driver.start_session(cap)
   try:
       b.get('https://whatismyip.com')
   except Exception as e:
       print(e)

p.s. selenium.webdriver.common.proxy.Proxy.add_to_capabilities() may also be used when specifying proxy (so you do not need to use the c dict above.)

Upvotes: 4

Florent B.
Florent B.

Reputation: 42518

To set a proxy on the fly with Firefox:

def set_proxy(driver, http_addr='', http_port=0, ssl_addr='', ssl_port=0, socks_addr='', socks_port=0):

    driver.execute("SET_CONTEXT", {"context": "chrome"})

    try:
        driver.execute_script("""
          Services.prefs.setIntPref('network.proxy.type', 1);
          Services.prefs.setCharPref("network.proxy.http", arguments[0]);
          Services.prefs.setIntPref("network.proxy.http_port", arguments[1]);
          Services.prefs.setCharPref("network.proxy.ssl", arguments[2]);
          Services.prefs.setIntPref("network.proxy.ssl_port", arguments[3]);
          Services.prefs.setCharPref('network.proxy.socks', arguments[4]);
          Services.prefs.setIntPref('network.proxy.socks_port', arguments[5]);
          """, http_addr, http_port, ssl_addr, ssl_port, socks_addr, socks_port)

    finally:
        driver.execute("SET_CONTEXT", {"context": "content"})

Usage:

 driver = webdriver.Firefox()

 set_proxy(driver, http_addr="212.35.56.21", http_port=8080)

 driver.get("http://....")

 set_proxy(driver, http_addr="212.35.56.22", http_port=8888)

 driver.get("http://....")

Upvotes: 5

Bob Kimani
Bob Kimani

Reputation: 1164

This is a slightly old question. But it is actually possible to change the proxies dynamically thru a "hacky way" I am going to use Selenium JS with Firefox but you can follow thru in the language you want.

Step 1: Visiting "about:config"

driver.get("about:config");

Step 2 : Run script that changes proxy

var setupScript=`var prefs = Components.classes["@mozilla.org/preferences-service;1"]
.getService(Components.interfaces.nsIPrefBranch);

prefs.setIntPref("network.proxy.type", 1);
prefs.setCharPref("network.proxy.http", "${proxyUsed.host}");
prefs.setIntPref("network.proxy.http_port", "${proxyUsed.port}");
prefs.setCharPref("network.proxy.ssl", "${proxyUsed.host}");
prefs.setIntPref("network.proxy.ssl_port", "${proxyUsed.port}");
prefs.setCharPref("network.proxy.ftp", "${proxyUsed.host}");
prefs.setIntPref("network.proxy.ftp_port", "${proxyUsed.port}");
                  `;    

//running script below  
driver.executeScript(setupScript);

//sleep for 1 sec
driver.sleep(1000);

Where use ${abcd} is where you put your variables, in the above example I am using ES6 which handles concatenation as shown, you can use other concatenation methods of your choice , depending on your language.(The SetupScript is a string containing the script to be runned enclosed by ``)

Step 3: : Visit your site

driver.get("https://whatismyip.com");

Explanation:the above code takes advantage of Firefox's API to change the preferences using JavaScript code.

Upvotes: 14

Related Questions