Reputation: 6594
I've implemented a scraper that uses HTTPS protocol. To avoid captchas and HTTP 429 error I'm using proxies. That doesn't however guarantee no captchas, so sometimes I need to unblock a proxy entering the captcha manually. I'm using Windows, so I'm changing LAN settings from the Internet Properties > Connection tab, opening the site in the browser, and entering the captcha. This procedure is a little time consuming as I need to open menus, click mouse many times, copy/paste IP:port, etc. I wish to develop a script that does part of this job automatically: it should set the proxy IP/port, leaving me just the trouble of entering captcha.
As I have found already, this setting is located in the registry: HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ProxyServer
, and changing a registry value is not a problem. This change however doesn't take effect immediately, and I still have to open the dialog to apply the changes.
My first question is how to apply the changes automatically (something that I can do from the Python/C++/bash, etc.) The wider question is whether there is a more reliable method to solve the problem: not to touch registry explicitly but to use a tool that makes changes/applies the settings, etc?
Upvotes: 1
Views: 13695
Reputation: 1979
There is a Powershell command to turn the proxy on, and then another command to set the proxy as well.
To turn on the proxy,
Set-ItemProperty -path "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings" ProxyEnable -value 1
To turn off the proxy, set the value to 0 and run the Powershell command.
To set a proxy and port,
Set-ItemProperty -path "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings" ProxyServer -value (proxy to be used):(port to be used)
From a superuser article,
Unfortunately, there is no easy way. As you’ve noticed, you’re missing the magic “read those settings now” command:
InternetSetOption(NULL, INTERNET_OPTION_SETTINGS_CHANGED, NULL, NULL) InternetSetOption(NULL, INTERNET_OPTION_REFRESH, NULL, NULL)
Of course, you can’t just call a C function from cmd. There is, however, a (relatively) viable way to do it with PowerShell.
You can call the script to run from Python to run these commands, after changing the registry, if you want minimal change.
Upvotes: 0