CODEWITHSUNDEEP
python-2.7
user5039912
user5039912

Reputation:

how to Implement proxy in python

I want to implement proxy in my code snippet. How can I achieve this?

try:
     raw = self.session.get(self.BASE_URL + '/archive').text
except:
     logging.info('Error with pastebin')
     raw = None
     sleep(5)
results = BeautifulSoup(raw).find_all(lambda tag: tag.name == 'td' and tag.a and '/archive/' not in tag.a['href'] and tag.a['href'][1:])

Upvotes: 0

Views: 160

Answers (1)

user5043534
user5043534

Reputation:

You can import urllib and try doing as below to implement proxy

try:
    raw = self.session.get(self.BASE_URL + '/archive').text

except:
    proxy_support = urllib2.ProxyHandler({"http":"http://10.62.51.201:3128"})
    opener = urllib2.build_opener(proxy_support)
            urllib2.install_opener(opener)

    #results = BeautifulSoup(helper.download(urllib2.urlopen(self.BASE_URL).geturl())+  '/archive').find_all(lambda tag: tag.name == 'td' and tag.a and '/archive/' not in tag.a['href'] and tag.a['href'][1:])
    myurl = urllib2.urlopen(self.BASE_URL).geturl()
    raw = self.session.get(myurl + '/archive').text

results = BeautifulSoup(raw).find_all(lambda tag: tag.name == 'td' and tag.a and '/archive/' not in tag.a['href'] and tag.a['href'][1:])

Upvotes: 1

Related Questions