CODEWITHSUNDEEP
pythonpython-2.7proxyurllib2mechanize
Ryflex
Ryflex

Reputation: 5769

Using python mechanize and also urllib2 with random proxy support

I've got some http proxies for example:

123.123.123.123:2312
121.111.3.89:8080
111.133.1.111:23810
114.113.1.113:23812
111.133.1.114:23810

They all have the same username and password: testuser and testpass

I'm trying to incorporate random proxy support for the following:

import urllib2
import httplib
def check():
    try:
        urllib2.urlopen("", timeout = 60)
        return True
    except (IOError, httplib.HTTPException):
        return False

Also trying to incorporate it into the following:

import mechanize
def gethtml():
        post_url = ""
        browser = mechanize.Browser()
        browser.set_handle_robots(False)
        browser.addheaders = [('User-agent', 'Firefox')]

        try:
            html = browser.open(post_url).read()
        except Exception:
            return

And also onto the similar:

import mechanize
def check2():
    post_url = ""
    browser = mechanize.Browser()
    browser.set_handle_robots(False)
    browser.addheaders = [('User-agent', 'Firefox')]
    parameters = {'page' : '1',
                  'sortorder' : 'asc'
                 }

    data = urllib.urlencode(parameters)

    try: 
        trans_array = browser.open(post_url,data).read().decode('UTF-8')
    except Exception:
        return

My biggest problem is everything that I've tried, I get the following two errors:

httperror_seek_wrapper: HTTP Error 407: Proxy Authentication Required

HTTPError: HTTP Error 407: Proxy Authentication Required

Is anyone able to help me create some working examples, it would be much appreciated.

Upvotes: 2

Views: 846

Answers (1)

crennie
crennie

Reputation: 674

"Random proxy support" with mechanize? Would that just be regular mechanize proxy stuff, but with a randomly-picked proxy from your list of proxies?

If so, you can try this:

import mechanize
import random

proxies = [
    '123.123.123.123:2312',
    '121.111.3.89:8080'
    '111.133.1.111:23810',
    '114.113.1.113:23812',
    '111.133.1.114:23810',
    ]
rand_proxy = random.choice(proxies)

browser = mechanize.Browser()
browser.set_proxies({'http': rand_proxy})
browser.add_proxy_password('testuser', 'testpass')

Upvotes: 1

Related Questions