CODEWITHSUNDEEP
dnsweb-crawlerbots
StormShadow
StormShadow

Reputation: 1627

What's the best way to programmatically scan domains under a given TLD?

Not sure if I want to just do a kind of brute-force thing, or a dictionary attack (though this would require a Romanised dictionary of every word in the target language), but I want to scan area websites (area meaning in my country) and I don't want to just whitelist everything. Is there a better way to do this?

Upvotes: 1

Views: 204

Answers (1)

user3967089
user3967089

Reputation:

If you have a legitimate need for a list of all zones in a TLD, contact the registry for that TLD and ask. How willing they will be to help you varies enormously, so without knowing which TLD you're thinking of it's impossible to guess how viable this way is.

If the TLD uses DNSSEC with NSEC, you can walk the zone by following the NSEC chain. This is the best way if you can't get a file to download, and the fact that a zone is using NSEC is implicit permission to do so.

If you can't do either of the above, you're down to guessing. You're also working against the wishes of the registry, so be prepared to have your server get blocked from even talking to the TLD's name servers. There are about 6.26*10^98 possible names directly under each TLD, so you'll need to send quite a few queries.

Also note that "web servers in my country" is not a very well defined concept. Does that mean all sites with domain names in your country's ccTLD? All sites hosted on servers that are physically in your country? Sites intended for people in your country, no matter their domain name or hosting location?

Upvotes: 1

Related Questions