CODEWITHSUNDEEP
akkaakka-clusterakka-remote-actorakka-remoting
blaha
blaha

Reputation: 2755

Akka Seed Node on AWS drops all its messages because they don't match inbound addresses

I've discovered a lot of lesser-documented configuration options to use, but I can't seem to configure my seed node such that it can both contact itself and be contacted by other nodes.

At the moment, I have it configured so:

akka {
  cluster.multi-data-center.self-data-center = asia  
  remote.netty.tcp.hostname = "xxx.xx.xx.5"  
  remote.netty.tcp.public-hostname = "xx.xx.xxx.51"  
  cluster.seed-nodes = ["akka.tcp://[email protected]:1551"]  
  enforce-ip-family = false
  dns-use-ipv6 = false
}

In this configuration, the node can connect to itself as seed node and function by itself, but other nodes that try to contact it at the public-hostname have their messages dropped:

2018-01-25 19:29:56,934 [ERROR]: akka.remote.EndpointWriter in application-akka.actor.default-dispatcher-5 - dropping message [class akka.actor.ActorSelectionMessage] for non-local recipient [Actor[akka.tcp://[email protected]:1551/]] arriving at [akka.tcp://[email protected]:1551] inbound addresses are [akka.tcp://[email protected]:1551]

My research indicated that the public-hostname configuration exists to solve this problem. Maybe not? I tried the opposite, setting the hostname to the public IP and configuring the bind-hostname to the IP that enables the seed node to connect to itself:

akka {
  cluster.multi-data-center.self-data-center = asia  
  remote.netty.tcp.hostname = "xx.xx.xxx.51"  
  remote.netty.tcp.bind-hostname = "xxx.xx.xx.5"  
  cluster.seed-nodes = ["akka.tcp://[email protected]:1551"]  
  enforce-ip-family = false
  dns-use-ipv6 = false
}

Then I encounter the same paradox in the opposite direction:

2018-01-25 19:39:08,207 [WARN ]: akka.cluster.JoinSeedNodeProcess in application-akka.actor.default-dispatcher-4 - Couldn't join seed nodes after [3] attempts, will try again. seed-nodes=[akka.tcp://[email protected]:1551]

2018-01-25 19:38:48,168 [ERROR]: akka.remote.EndpointWriter in application-akka.actor.default-dispatcher-6 - dropping message [class akka.actor.ActorSelectionMessage] for non-local recipient [Actor[akka.tcp://[email protected]:1551/]] arriving at [akka.tcp://[email protected]:1551] inbound addresses are [akka.tcp://[email protected]:1551]

The seed node is now unable to connect to itself, because xx.xx.xxx.51 has taken over as the inbound address.

I have also attempted to use both public-hostname and bind-hostname, with no success.

Upvotes: 1

Views: 551

Answers (1)

blaha
blaha

Reputation: 2755

I was able to get my seed to bind to itself and be available to the other nodes by using the external .51 address as the configured seed for ALL NODES (including the seed itself), as well as the akka.remote.netty.tcp.hostname for the seed. I also set the internal .5 address as the bind-hostname on the seed. No public-hostname was needed.

Upvotes: 2

Related Questions