CODEWITHSUNDEEP
pythonauthenticationseleniumproxy
USHA
USHA

Reputation: 123

How to run selenium web driver behind a proxy server which needs authentication in python

At present this is my code, but webDriver is showing a pop-up to enter proxy credentials and I don't want this annoying situation, This is not the first time the same question appeared in stackoverflow, but no one replied with a proper answer.

I tried google to find a solution for this problem. I came to know the solution in java, but i dont know how we do it in python.

    PROXY_HOST = "65.49.1.59"
    PROXY_PORT = 60099
    fp = webdriver.FirefoxProfile()
    # Direct = 0, Manual = 1, PAC = 2, AUTODETECT = 4, SYSTEM = 5

    print " im in parse_details"

    fp.set_preference("network.proxy.type", 1)
    fp.set_preference('network.http.phishy-userpass-length', 255)

    fp.set_preference("network.proxy.http", PROXY_HOST)
    fp.set_preference("network.proxy.http_port", PROXY_PORT)
    fp.set_preference("network.proxy.ftp", PROXY_HOST)
    fp.set_preference("network.proxy.ftp_port", PROXY_PORT)
    fp.set_preference("network.proxy.ssl", PROXY_HOST)
    fp.set_preference("network.proxy.ssl_port", PROXY_PORT)
    #fp.set_preference("network.proxy.user_name", 'someusername')
    #fp.set_preference("network.proxy.password", 'somepassword')
    fp.set_preference("network.proxy.no_proxies_on", "") # set this value as desired

    self.driver = webdriver.Firefox(firefox_profile=fp)
    self.driver.get("http://www.whatismyip.com/")

These below statements are guessed by me, and I am not sure whether their syntax is correct or not, even i tried to find out in selenium documentation, but no help. Would you guys throw some light on this.

    #fp.set_preference("network.proxy.user_name", 'someusername')
    #fp.set_preference("network.proxy.password", 'somepassword')

p.s. The same question asked here Selenium using Python: enter/provide http proxy password for firefox

Upvotes: 12

Views: 18895

Answers (3)

Gaurav Sharma
Gaurav Sharma

Reputation: 349

I know its pretty late replying for your question but recently I started working with Python, and was trying to do the same and did something like this to handle this situation.

To run selenium web driver behind proxy server

  1. Need to create a firefox profile, where "autoauth" add-on should be installed.
  2. Try to save proxy server username & password by hitting a URL manually.
  3. Firefox profile will save the credential of proxy server with help of autoauth
  4. In script call that particular Firefox profile.
  5. Set all the preferences to define proxy server details.
  6. Assign Firefox profile in to browser's instance
  7. Hit any URL, Below is the running example

P.S. : Remove all proxy setting from internet options, script will use it automatically

So technically here you will not send proxy username & password, you'll save those credential in firefox and invoke that particular firefox profile.

Hope you have already solved your problem long back, but in case its still there, this might help you. :)

Upvotes: 3

Mike
Mike

Reputation: 2153

Selenium can't handle with basic authentication neither it works well with popups. But this problem is resolvable. As a solution that worked to me (I found it here) is to add a browser extension that does the authentication for Selenium. It's quite simple. Here is how it works for Chrome and Python:

  1. Create a zip file proxy.zip containing two files:

background.js

var config = {
    mode: "fixed_servers",
    rules: {
      singleProxy: {
        scheme: "http",
        host: "YOU_PROXY_ADDRESS",
        port: parseInt(YOUR_PROXY_PORT)
      },
      bypassList: ["foobar.com"]
    }
  };

chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});

function callbackFn(details) {
    return {
        authCredentials: {
            username: "YOUR_PROXY_USERNAME",
            password: "YOUR_PROXY_PASSWORD"
        }
    };
}

chrome.webRequest.onAuthRequired.addListener(
        callbackFn,
        {urls: ["<all_urls>"]},
        ['blocking']
);

Don't forget to replace YOUR_PROXY_* to your settings.

manifest.json

{
    "version": "1.0.0",
    "manifest_version": 2,
    "name": "Chrome Proxy",
    "permissions": [
        "proxy",
        "tabs",
        "unlimitedStorage",
        "storage",
        "<all_urls>",
        "webRequest",
        "webRequestBlocking"
    ],
    "background": {
        "scripts": ["background.js"]
    },
    "minimum_chrome_version":"22.0.0"
}
  1. Add the created proxy.zip as an extension

Python Code:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_extension("proxy.zip")

driver = webdriver.Chrome(executable_path='chromedriver.exe', chrome_options=chrome_options)
driver.get("http://google.com")
driver.close()

That's it. For me that worked like a charm. If you need to create proxy.zip dynamically or need PHP example then go to the original post

Upvotes: 15

steiryx
steiryx

Reputation: 81

Refer to this path on how to access the firefox profile: http://software-testing-tutorials-automation.blogspot.jp/2014/05/how-to-create-and-use-custom-firefox.html

Basically you need to create a profile and use that profile in firefox. Now the behaviour of your profile will also be the same when you run in via selenium. So if you're profile will prompt proxy auth, then it will also display when you open it via selenium. So you have to find a way by using add-ons to hide the authentication using that profile. After you have successfully done that, it will be the same now with Selenium.

Upvotes: 0

Related Questions