spring-cloud-netflix eureka aws deployment

Question

I've been scouring the internet looking for guidance on the proper way to deploy spring-cloud-netflix eureka server to aws. We've been using docker quite a bit for microservices using both spring-cloud and nodejs, so I've been trying to deploy my spring-boot spring-cloud eureka app to ECS. There's very limited information in the spring-cloud documentation, but I found some fairly useful information here: https://www.slideshare.net/ToddMiller34/spring-cloud-into-production. Based on this, I currently have the following:

Standard spring-cloud-starter-eureka-server (spring-cloud Camden.SR5, spring-boot 1.5.1) with

@Bean
@Autowired
@Profile("aws")
public EurekaInstanceConfigBean eurekaInstanceConfigBean(InetUtils inetUtils) {
    EurekaInstanceConfigBean config = new EurekaInstanceConfigBean(inetUtils);
    AmazonInfo info = AmazonInfo.Builder.newBuilder().autoBuild("eureka");
    config.setHostname(info.get(AmazonInfo.MetaDataKey.publicHostname));
    config.setIpAddress(info.get(AmazonInfo.MetaDataKey.publicIpv4));
    config.setDataCenterInfo(info);
    return config;      
}

relevant pieces of configuration from cloud config repo:

eureka:
  instance:
    non-secure-port: ${server.port}
  environment: test
  client:
    eurekaServerDNSName: test.mydomain.com
  datacenter: cloud
  client:
    region: us-west-2
    registerWithEureka: true
    fetchRegistry: true
    eurekaServerURLContext: eureka
    eurekaServerPort: 8761
    useDnsForFetchingServiceUrls: true

aside: I was surprised that eureka.client.region was required, but without it the dns fetch for service urls went to txt.us-east-1... by default.

route53 txt records in private hosted zone:

txt.us-west-2.test.mydomain.com = "us-west-2a.test.mydomain.com" "us-west-2b.test.mydomain.com"
txt.us-west-2a.test.mydomain.com = "ec2-123-456-789-123.us-west-2.compute.amazonaws.com"
txt.us-west-2b.test.mydomain.com = "ec2-987-654-321-123.us-west-2.compute.amazonaws.com"

I had some eureka.server.bindingStrategy set to 'route53', but this doesn't work (gives me an error about not being able to find a free domain in localhost from Route53Binder line 135).

I got the ec2-### hostnames from the ec2 container instances, and things seem to be working, but I'm very concerned that these hostnames are going to change anytime the container instances get restarted. I've read about route53 binding, but that doesn't seem to be compatible with fetching service urls from dns and also doesn't seem well supported by netflix themselves (I've read comments on bugs by netflix engineers that the route53 binder was community-donated code, and that they don't use it at netflix). I've also read about EIP binding, but I'm not sure how to use that with ECS. Do I just create as many EIPs as I have eureka servers, put those in the route53 txt records and eureka will automatically associate? This all seems sorely lacking in documentation.

Answer 1

answering my own question in case someone else comes across this and it's helpful.

Do I just create as many EIPs as I have eureka servers, put those in the route53 txt records and eureka will automatically associate?

Yes, that's exactly what I did, and it works. I created 2 EIPs because I only have 1 eureka instance in each of 2 AZs in my environment. The EIPs are just ip addresses. I then put those in my route53 txt records, like this:

txt.us-west-2.test.mydomain.com = "us-west-2a.test.mydomain.com" "us-west-2b.test.mydomain.com" 
txt.us-west-2a.test.mydomain.com = "ec2-123-456-789-123.us-west-2.compute.amazonaws.com" 
txt.us-west-2b.test.mydomain.com = "ec2-987-654-321-123.us-west-2.compute.amazonaws.com"

In this case, the ec2-###-###-###-### entries are hostnames based on the IP addresses from EIP. For example, if the EIP address is 123.456.789.123, and you're in us-west-2, then use ec2-123-456-789-123.us-west-2.compute.amazonaws.com.

Eureka in ECS starts up and because my eureka client settings are set to fetch from dns, it finds the EIP addresses and automatically binds to one in the appropriate zone. Eureka clients in microservice applications need the same eureka client config, so they can find the same eureka server hostnames.

Relevant sections of cloud config yml files:

application.yml
==================
eureka:
  datacenter: cloud
  environment: production
  client:
    fetchRegistry: true
    eurekaServerURLContext: eureka
    region: us-west-2
    eurekaServerPort: 8761
    useDnsForFetchingServiceUrls: true
    eurekaServerDNSName: test.mydomain.com

eureka-server.yml
======================
eureka:
  server:
    renewalPercentThreshold: 0.49 

Note that eurekaServerDNSName is used in the TXT record lookup routine. In this mode (eureka.client.useDnsForFetchingServiceUrls) the spring-cloud eureka server and clients seems to use txt.${eureka.client.region}.${eureka.client.eurekaServerDNSName}

to find this record: txt.us-west-2.test.mydomain.com = "us-west-2a.test.mydomain.com" "us-west-2b.test.mydomain.com"

then uses the results of that dns lookup to find the other 2 that actually point to EIP-based domain names: txt.us-west-2a.test.mydomain.com = "ec2-123-456-789-123.us-west-2.compute.amazonaws.com" txt.us-west-2b.test.mydomain.com = "ec2-987-654-321-123.us-west-2.compute.amazonaws.com"

and that's how they populate the eureka server cluster node members. If you needed to add another server, you'd provision another EIP, add to the route53 records, and start a new server in ECS that would bind to the address you just added.