python 3 - urllib issue

Question

I'm using python 3.3.0 in Windows 7. I have two files: dork.txt and fuzz.py

dork.txt contains following:

/about.php?id=1
/en/company/news/full.php?Id=232
/music.php?title=11

fuzz.py contains following:

srcurl = "ANY-WEBSITE"
drkfuz = open("dorks.txt", "r").readlines()
print("\n[+] Number of dork names to be fuzzed:",len(drkfuz))

for dorks in drkfuz:
    dorks = dorks.rstrip("\n")
    srcurl = "http://"+srcurl+dorks

    requrl = urllib.request.Request(srcurl) 

    #httpreq = urllib.request.urlopen(requrl)

    # Starting the request
    try:
        httpreq = urllib.request.urlopen(requrl)
    except urllib.error.HTTPError as  e:
        print ("[!] Error code: ",  e.code)
        print("")
        #sys.exit(1)

    except urllib.error.URLError as  e:
        print ("[!] Reason: ",  e.reason)
        print("")
        #sys.exit(1)  

    #if e.code != 404:
    if httpreq.getcode() == 200:
        print("\n*****srcurl********\n",srcurl)
        return srcurl

So, when I enter the correct website name which has /about.php?id=1, it works fine. But when I provide the website which has /en/company/news/full.php?Id=232, it first prints Error code: 404 and then gives me the following error: UnboundLocalError: local variable 'e' referenced before assignment or UnboundLocalError: local variable 'httpreq' referenced before assignment

I can understand that if the website doesn't have the page which contains /about.php?id=1, it gives Error code: 404 but why it's not going back in the for loop to check the remaining dorks in the text file??? Why it stops here and throws an error?

I want to make a script to find out valid page from just a website address like: www.xyz.com

Answer 1

srcurl = "ANY-WEBSITE"
drkfuz = open("dorks.txt", "r").readlines()
print("\n[+] Number of dork names to be fuzzed:",len(drkfuz))

for dorks in drkfuz:
    dorks = dorks.rstrip("\n")
    srcurl = "http://"+srcurl+dorks

    try:
        requrl = urllib.request.Request(srcurl)
        if requrl != None and len(requrl) > 0:
            try:
                httpreq = urllib.request.urlopen(requrl)
                if httpreq.getcode() == 200:
                    print("\n*****srcurl********\n",srcurl)
                    return srcurl
            except:
                # Handle exception
                pass
    except:
        # Handle your exception
        print "Exception"

Untested code, but it will work logically.

Answer 2

When the line urllib.request.urlopen(requrl) expression throws an exception, the variable httpreq is never set. You could set it to None before the try statement, then test if it is still None afterwards:

httpreq = None

try:
    httpreq = urllib.request.urlopen(requrl)

# ...

if httpreq is not None and httpreq.getcode() == 200: