CODEWITHSUNDEEP
pythonseleniumgoogle-chrome-extension
John Doe
John Doe

Reputation: 81

Selenium chrome hanging every other time

My selenium chrome script hangs every other time it is run, and it only does this when it is run with an extension, however I have no idea why.

I have recently, and what seems to be out of nowhere, encountered a problem that is seriously confusing me. I have a python selenium script run through Chrome that utilizes a proxy server, as it uses a username/password combination to authenticate and is not just authenticated through IP I have made a chrome extension that the driver loads before starting a new session.

Below is the script:

import selenium
import selenium.webdriver.common.proxy
import selenium.webdriver.common.desired_capabilities

chrome_options = selenium.webdriver.ChromeOptions()
chrome_options.add_argument("disable-infobars")
chrome_options.add_argument('--ignore-certificate-errors')
chrome_options.add_argument('--ignore-ssl-errors')


prefs = {"profile.default_content_setting_values.notifications" : 2, "profile.managed_default_content_settings.images": 2, "profile.default_content_settings.images":2, 'disk-cache-size': 4096 }
chrome_options.add_experimental_option("prefs",prefs)

add_log_prefs = selenium.webdriver.common.desired_capabilities.DesiredCapabilities.CHROME
add_log_prefs['loggingPrefs'] = { 'browser':'ALL' }

chrome_options.add_argument('--load-extension='+proxy_extension_path)

driver = selenium.webdriver.Chrome(chrome_driver_path,
                                   options=chrome_options,
                                   desired_capabilities=add_log_prefs)

driver.set_window_size(window_size_w, window_size_h)

driver.get('https://www.google.com/')

Below is the extension:

Manifest.json:

{
    "version": "1.0.0",
    "manifest_version": 1,
    "name": "Chrome Proxy",
    "permissions": [
        "proxy",
        "tabs",
        "unlimitedStorage",
        "storage",
        "<all_urls>",
        "webRequest",
        "webRequestBlocking"
    ],
    "background": {
        "scripts": ["background.js"]
    },
    "minimum_chrome_version":"1.0.0"
}

background.js:

var config = {
        mode: "fixed_servers",
        rules: {
          singleProxy: {
            scheme: "http",
            host: "HOST",
            port: parseInt(PORT)
          },
          bypassList: ["localhost"]
        }
      };

chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});

function callbackFn(details) {
    return {
        authCredentials: {
            username: "USERNAME",
            password: "PASSWORD"
        }
    };
}

chrome.webRequest.onAuthRequired.addListener(
            callbackFn,
            {urls: ["<all_urls>"]},
            ['blocking']
);

The issue I am experiencing is that when i start the script and try to load a website through get e.g. driver.get("https://www.google.com/"), it will navigate to the website every other time i start the program and every other time it will timeout. When it times out, I can manually go in and navigate to the website myself, so it seems like it is just hanging there.

This happens even when I re-run the script and scrub it for any loaded variables etc., but the fact that it happens every other time to me indicates that there is somehow some left-over setting that gets cleaned after the script times out and then resat when after it manages to navigate to the webaddress.

Edit: Additionally I have tried to add a profile and alter that, however this does not help my case either. I thought about using the proxy server differently but I cant find any way to use a proxy server with password/name besides an extension.

I am completely at a loss and have been working on this problem for hours on end at this point, so any suggestions would be much appreciated.

Upvotes: 1

Views: 668

Answers (1)

supputuri
supputuri

Reputation: 14135

Using a custom chrome profile is one of the solutions that resolve this issue. Don't ask me how and why :-) still have to investigate on that part.

But for now, if you want to continue with your scripts then add the below line to your chrome options.

 options.add_argument(r"--user-data-dir=path\to\chrome\user data\any_new_profile_name")
 # below is the sample
 options.add_argument(r"--user-data-dir=C:\Users\xxxx\AppData\Local\Google\Chrome\User Data\ChromeAutoProfile")

You don't have to create the new chrome profile, script will create the profile first time if it does not exist.

Upvotes: 1

Related Questions