CODEWITHSUNDEEP
pythonproxyurllib2socks
Mike
Mike

Reputation: 483

How can I use a SOCKS 4/5 proxy with urllib2?

How can I use a SOCKS 4/5 proxy with urllib2 to download a web page?

Upvotes: 48

Views: 50926

Answers (3)

panweizeng
panweizeng

Reputation: 796

You can use SocksiPy module. Simply copy the file "socks.py" to your Python's lib/site-packages directory, and you're ready to go.

You must use socks before urllib2. (Try it pip install PySocks )

For example:

import socks
import socket
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 8080)
socket.socket = socks.socksocket
import urllib2
print urllib2.urlopen('http://www.google.com').read()

You can also try pycurl lib and tsocks, for more detail, click on here.

Upvotes: 68

sw.
sw.

Reputation: 3231

Adding an alternative to pan's answer when you need to use many different proxies at the same time.

In that case you need to create an opener like you do with a http proxy. There is a code available in GitHub https://gist.github.com/869791

opener = urllib2.build_opener(SocksiPyHandler(socks.PROXY_TYPE_SOCKS4, 'localhost', 9999))
print opener.open('http://www.whatismyip.com/automation/n09230945.asp').read()

Upvotes: 21

fviktor
fviktor

Reputation: 2958

Since SOCKS is a socket level proxy, you have to replace the socket object used by urllib2. Please take a look a this solution. If monkey patching is not good enough for you, then you can try to subclass or copy-modify the code from the urllib2 standard library.

Upvotes: 4

Related Questions