CODEWITHSUNDEEP
pythontwisted
Sebastian Wiesinger
Sebastian Wiesinger

Reputation: 823

Asynchronous DNS resolver testing

I want to test a large number of IPs to look for open DNS resolvers. I'm trying to find the most efficient way to parallelize this. At the moment I'm trying to accomplish this with twisted. I want to have 10 or 20 parallel threads sending a query to avoid blocking trough timeouts.

Twisted has a DNSDatagramProtocol that seems suitable but I just can't figure out how to put it together with the twisted "reactor" and "threads" facilities to make it run efficiently.

I read a lot of the twisted documentation but I'm still not sure what would be the best way to do it.

Could someone give an example how this can be accomplished?

Upvotes: 2

Views: 2484

Answers (2)

Darek
Darek

Reputation: 2911

Try gevent, spawn many greenlets to do a DNS resolution. Also gevent has a nice DNS resolution API : http://www.gevent.org/gevent.dns.html

They have even an example: https://github.com/gevent/gevent/blob/master/examples/dns_mass_resolve.py

Upvotes: -1

Jean-Paul Calderone
Jean-Paul Calderone

Reputation: 48325

Here's a quick example demonstrating the Twisted Names API:

from sys import argv
from itertools import cycle
from pprint import pprint

from twisted.names import client
from twisted.internet.task import react
from twisted.internet.defer import gatherResults, inlineCallbacks

def query(reactor, server, name):
    # Create a new resolver that uses the given DNS server
    resolver = client.Resolver(
        resolv="/dev/null", servers=[(server, 53)], reactor=reactor)
    # Use it to do an A request for the name
    return resolver.lookupAddress(name)

@inlineCallbacks
def main(reactor, *names):
    # Here's some random DNS servers to which to issue requests.
    servers = ["4.2.2.1", "8.8.8.8"]

    # Handy trick to cycle through those servers forever
    next_server = cycle(servers).next

    # Issue queries for all the names given, alternating between servers.
    results = []
    for n in names:
        results.append(query(reactor, next_server(), n))
    # Wait for all the results
    results = yield gatherResults(results)
    # And report them
    pprint(zip(names, results))

if __name__ == '__main__':
    # Run the main program with the reactor going and pass names
    # from the command line arguments to be resolved
    react(main, argv[1:])

Upvotes: 3

Related Questions