CODEWITHSUNDEEP
pythonhttpsrequestproxies
Nazariy
Nazariy

Reputation: 727

Does urllib.getproxies work with https websites?

I am trying to use proxies to scrape some informations off a website, however I have heard that urlib.getProxies() doesn't work, especially for https websites.Here is my code: 

page = requests.get(url,proxies = urllib.getproxies())

So does this actually work and is there a way to find out if it is requesting through proxies, instead of my ip address.By the way I am requesting a https website. Thanks for your help ahead of time.

Upvotes: 0

Views: 2226

Answers (1)

will
will

Reputation: 10650

this question & answer explains exactly what your problem might be.

The proxyDict you're supplying to requests.get(... proxies = X) needs to have a proxy set for the specific protocol of the request you're making. That is, if you're making an https requests, you need to have proxyDict["https"] set to something.

What does urllib.getproxies() return for you? Is there anything set for the https protocol?

Upvotes: 1

Related Questions