AWS Route53 Private Hosted Zones and Reverse Lookup

Question

Working with AWS as our Cloud Provider... our platform engineering team is wanting to leverage Route53 to deliver our DNS Solution for both Public and Private Hosted Zones.

Also, we've been working with Custom DHCP Options so that we can provide custom hostnames to our EC2 Instances, but are presently encountering an issue with Reverse DNS Lookup to reference the correct Hostname...

SCENARIO When we provision a new EC2 Instance in an environment (e.g. DEV) we want to provision it with a hostname using our custom domain name... ip-10-100-1-10.dev.example.net vs ip-10-100-1-10.ec2.internal and still be able to leverage AmazonProvidedDNS

Here is an example of our DHCP Options, whereby we specify a custom domain and point to AmazonProvidedDNS...

EXAMPLE DHCP OPTIONS
domain-name = dev.example.net
domain-name-servers = AmazonProvidedDNS

For Route53, we have created two private hosted zones... one for forward lookup and one for reverse lookup...

# Forward Zone  dev.example.net.
ip-10-100-1-10.dev.example.net.   A      10.100.1.10

# Reverse Zone  1.100.10.in-addr.arpa.
10.1.100.10.in-addr.arpa.   PTR    ip-10-100-1-10.dev.example.net.

Once, these artifacts are in place... DHCP Options, Forward Zone, Reverse Zone... and we perform a Reverse Lookup by IP we are not getting the desired result... i.e. the custom hostname we've specified in our Reverse Zone, doesn't appear to be propagating...

nslookup 10.100.1.10
Server:     10.100.0.2
Address:    10.100.0.2#53

Non-authoritative answer:
10.1.100.10.in-addr.arpa    name = ip-10-100-1-10.ec2.internal.

As you can see above, we still get the Amazon ec2.internal domain returned.

Is there a way to overwrite this behavior? We have noted that the TTL for the Private Zones is set at 48 hours (172800). So, we continue to test.

Our objective is that we are able to benefit from Route53 DNS and also use Custom DHCP Options. Thus far, our attempts at configuring Route53 Private Hosted Forward and Reverse Zones to achieve this result are not working as expected.

The only two options remaining appears to be an either/or...

1.) Use Default DHCP Options... AmazonDNS Domain (e.g. ec2.internal) and AmazonDNSProvider.

2.) Use our own Hosted DNS... updating DHCP Option to specify the Custom Domain and Point to our own Name Servers.

I've read thru a few posts on this topic related to Public Hosted Zones, but have not seen much on Private Hosted Zones. Thought I'd post this question out to get any comments or feedback on our approach.

Any comments, ideas, or suggestions would be greatly appreciated.

Cheers!

Answer 1

For Custom DHCP Options and using AmazonProvidedDNS here is the current configuration that we are using which is working:

VPC Configuration

DNS resolution = yes
DNS hostnames  = yes

DHCP Options

domain-name = dev.example.com
domain-name-servers = AmazonProvidedDNS

Route53 Private Hosted Zone (Forward)

# dev.example.com
ip-10-100-1-110.dev.example.com    A    10.100.1.110

Route53 Private Hosted Zone (Reverse)

# 1.110.10.in-addr.arpa.
110.1.100.10.in-addr.arpa.    PTR    ip-10-100-1-110.dev.example.com

Note: If you are working with VPC Peering Connections and Private Routes across VPC CIDR Blocks, you will want to be sure that you have correctly Associated those Peering VPCs with your Hosted Zones, so that DNS Resolution works for either VPC Environment.

Example: Let's say you have an operations vpc peers with your development vpc environment.

ops.example.com -> OPS VPC CIDR 10.10.0.0/16
dev.example.com -> DEV VPC CIDR 10.100.0.0/16

For DNS Resolution to work properly, in Route53 you will need to associate the VPCs to the Hosted Zones that they need to resolve.

ops.example.com zone associations
-> OPS VPC 10.10.0.0/16
-> DEV VPC 10.100.0.0/16

dev.example.com zone associations
-> DEV VPC 10.100.0.0/16
-> OPS VPC 10.10.0.0/16

This association will allow dev vpc hosts, applications, and services to successfully resolve ops.example.com DNS routes and vice versa. Once these associations are in place you will be able to resolve Forward and Reverse Zones using the VPC's Built-In Name Server.

nslookup 10.10.1.10
Server:     10.100.0.2
Address:    10.100.0.2#53

Non-authoritative answer:
10.1.10.10.in-addr.arpa   name = ip-10-10-1-10.dev.example.com.

If you are not doing so currently, I would highly recommend using a configuration and orchestration tool such as Terraform to codify, deploy, and version your AWS Platform from the network, security, and storage layers all the way up to databases, services, and applications.

Hope that this helps.

Cheers!