CODEWITHSUNDEEP
pythondnstwisted
AnkurVj
AnkurVj

Reputation: 8198

Does twisted epollreactor use non-blocking dns lookup?

It seems obvious that it would use the twisted names api and not any blocking way to resolve host names. However digging in the source code, I have been unable to find the place where the name resolution occurs. Could someone point me to the relevant source code where the host resolution occurs ( when trying to do a connectTCP, for example). I really need to be sure that connectTCP wont use blocking DNS resolution.

Upvotes: 2

Views: 618

Answers (2)

Glyph
Glyph

Reputation: 31860

It seems obvious, doesn't it?

Unfortunately:

  1. Name resolution is not always configured in the obvious way. You think you just have to read /etc/resolv.conf? Even in the specific case of Linux and DNS, you might have to look in an arbitrary number of files looking for name servers.
  2. Name resolution is much more complex than just DNS. You have to do mDNS resolution, possibly look up some LDAP computer records, and then you have to honor local configuration dictating the ordering between these such as /etc/nsswitch.conf.
  3. Name resolution is not exposed via a standard or useful non-blocking API. Even the glibc-specific getaddrinfo_a exposes its non-blockingness via SIGIO, not just a file descriptor you can watch. Which means that, like POSIX AIO, it's probably just a kernel thread behind your back anyway.

For these reasons, among others, Twisted defaults to using a resolver that just calls gethostbyname in a thread.

However, if you know that for your application it is appropriate to have DNS-only hostname resolution, and you'd like to use twisted.names rather than your platform resolver - in other words, if scale matters more to you than esoteric name-resolution use-cases - that is supported. You can install a resolver from twisted.names.client onto the reactor, appropriately configured for your application and all future built-in name resolutions will be made with that resolver.

Upvotes: 3

GP89
GP89

Reputation: 6730

I'm not massively familiar with twisted, I only recently started used it. It looks like it doesn't block though, but only on platforms that support threading.

In twisted.internet.base in ReactorBase it looks like it does the resolving through it's resolve method which returns a deferred from self.resolver.getHostByName.

self.resolver is an instance of BlockingResolver by default which does block, but it looks like that if the platform supports threading the resolver instance is replaced by ThreadedResolver in the ReactorBase._initThreads method.

Upvotes: 1

Related Questions