CODEWITHSUNDEEP
pythonseleniumproxy
Oleg Tarasenko
Oleg Tarasenko

Reputation: 9610

python: Selenium webdriver and hanging proxy issue

I am trying to understand how to handle cases when calls which are perfomed through proxies are hanging. For example I have this code:

def call_with_proxy(ip, port):
    profile = FirefoxProfile()
    profile.set_preference('network.proxy.type', 1)
    profile.set_preference('network.proxy.socks', ip)
    profile.set_preference('network.proxy.socks_port', port)
    profile.update_preferences()
    driver= webdriver.Firefox(profile)
    driver.get("http://somewebsite.com")

The proxy is taken from free proxies list here https://hidemyass.com/proxy-list/

Some times everythig works and I am getting the page I am requesting. But sometimes I am getting a blank firefox page (where I can see some elements of the website is being loaded, e.g. css), and this process lasts for a very long time. E.g. session is not being closed even after 10 minutes of such waiting time. I want to ask if there is a way, to automatically close browser if for example page is not loading for some time, or for example the test I am performing stopped execution (due to some reason related to proxies)

Upvotes: 1

Views: 2082

Answers (2)

boethius
boethius

Reputation: 438

Implement a heartbeat system using queues or other active runtime objects (ie/ weblistener). If you know the maximum runtime for the site script as a whole, you can use SE-Grid like functionality.

If you have variable times on the site, and are only worried about the initial load time, a heartbeat system is the only way I can think of.

Upvotes: 0

Erki M.
Erki M.

Reputation: 5072

In java we have:

    webDriver.manage().timeouts().pageLoadTimeout(30, TimeUnit.SECONDS);

From the doc:

pageLoadTimeout

WebDriver.Timeouts pageLoadTimeout(long time, java.util.concurrent.TimeUnit unit)

Sets the amount of time to wait for a page load to complete before throwing an error. If the timeout is negative, page loads can be indefinite.

Parameters: time - The timeout value. unit - The unit of time. Returns: A Timeouts interface.

Quick Googling shows:

webDriver.set_page_load_timeout(30)

for Python. Try this in try-catch (or try-except in your case)

Upvotes: 1

Related Questions