CODEWITHSUNDEEP
javadns
maugch
maugch

Reputation: 1318

Skip DNS query with Java if I have the IP

How do I skip a DNS request if I have a list of domains and their IP address? scans.io provides a list of domain and their IPs and I would like to use that list instead of sending a query to a DNS server. Is it ever possible without setting up my own dns server?

Edit: The reason is that I have a big file with a million of domain with their IP and I want to crawl them as fast as possible without getting banned from a DNS server.

Upvotes: 1

Views: 2387

Answers (3)

Stephen C
Stephen C

Reputation: 718787

If you use InetAddress.getByName(String) (javadoc) you can supply either a DNS name or the string representation of an IP address. In the latter case, it the method won't do an unnecessary DNS lookup.

What happens when an IP has more than one site? It's possible if I remember correctly, and giving just the IP shouldn't be enough.

A website is identified by a URL, not just a DNS name or IP address. The URL gives you the hostname or IP address, and also the protocol to use, the port number and the path. Any one of these can cause your HTTP requests to be directed / redirected to a different "site" ... whatever that means.

Basically, the process of connecting to a "site" starts as follows:

  1. The hostname (or IP address string) is converted to an address
  2. The port number is figured out (from the protocol and the optional explicit port number in the URL.
  3. The client opens a TCP connection to the IP / port determined above
  4. The URL is sent to the server listening on the IP / port in the HTTP start line.

If there are multiple "sites" on the same IP address + port (e.g. because multiple hostnames resolve to the same IP), then the server works out which "site" should respond based on the start line (and potentially, other HTTP headers).

If your goal is to reduce DNS lookups for HTTP interactions, then your best bet is to configure client-side DNS caching. Details are platform specific. Google can help you find the details.

You could also turn your big file of DNS entries into "/etc/hosts" syntax, and point the local resolver at it.

Upvotes: 1

absurdhero
absurdhero

Reputation: 356

You don't explain what you want to do with this data that you think requires a DNS lookup. If you are trying to make InetAddress objects in java, you can call InetAddress.getByAddress(host, addr) to construct an InetAddress without performing a DNS lookup.

See https://docs.oracle.com/javase/7/docs/api/java/net/InetAddress.html#getByAddress(java.lang.String,%20byte[])

To convert an address into a byte[], call InetAddress.getByName(addr).getAddress() if you know it is a valid address. It will not cause a DNS lookup. If you don't trust your data, you may want to write a small function to parse the address yourself and build a byte array.

Upvotes: 1

jarnbjo
jarnbjo

Reputation: 34313

There are several possibilities.

If you are not required to limit DNS lookups from your datafile to a specific Java process, you can add the list to your system's 'hosts' file. Where to find the file is OS specific and you have to consider that all processes running on the system would get the matching IP from the hosts file if they use the OS API calls to resolve an IP from a host name.

Another option is, if supported by your Java VM, to configure the host name resolver to use other data sources than a regular DNS lookup. Oracle's Java VM offers this possibility and you can find a documentation of the configuration in the section JNDI DNS service provider settings of the network properties documentation. Perhaps a little bit outdated, but this article also gives you a complete example on how to implement your own NameService using the API in sun.net.spi.nameservice.

Upvotes: 1

Related Questions