CODEWITHSUNDEEP
pythonconnectionwhoispywhois
Milano
Milano

Reputation: 18705

How to avoid program freezing when connecting to server

I have a little script which filters those domain names which are not registred yet. I use pywhois module. The problem is that it suddenly freeze and do nothing after several (sometimes hundreds) of requests. I think it is not a ban because I can run the program right after freeze and it works.

I would like to avoid this freezing. My idea is to count runtime of the function and if the time cross some line (for example 10 seconds) it repeats the code.

Do you have any advice how to avoid the freezing? Or the better way to check domains?

Here is the code:

for keyword in keywords:
        try:
            details = pythonwhois.get_whois(keyword+'.com')
        except Exception as e:
            print e
            continue
        if 'status' not in details.keys():
            print 'Free domain!'
            print keyword

Upvotes: 1

Views: 939

Answers (2)

amlweems
amlweems

Reputation: 66

This method is prone to change (if the underlying library changes), however, you can call internal socket functions to set a timeout for all pythonwhois network calls. For example:

TIMEOUT = 5.0 # timeout in seconds
pythonwhois.net.socket.setdefaulttimeout(TIMEOUT)
pythonwhois.get_whois("example.com")

Upvotes: 4

JDong
JDong

Reputation: 2304

Maybe you could try dnspython. It looks like you just want to check if a domain name is registered. For example:

import dns.resolver

for keyword in keywords:
    try:
        dns.resolver.query(keyword+'.com')
    except dns.resolver.NXDOMAIN:
        print(keyword+'.com is available!')

DNS resolver has a default timeout of 2 seconds. If you want to change that, you can make a new instance of dns.resolver.Resolver with a different timeout.

To make it multithreaded, a thread pool would be the best choice if you can use python3:

from multiprocessing import Pool

def check_keyword(keyword):
    try:
        dns.resolver.query(keyword+'.com')
    except dns.resolver.NXDOMAIN:
        # You probably want to change this to a return
        print(keyword+'.com is available!') 

if __name__ == '__main__':
    keywords = [...]
    p = Pool(5)
    print(p.map(check_keyword, keywords))

Upvotes: 0

Related Questions