CODEWITHSUNDEEP
apache-kafka
DoctaMag
DoctaMag

Reputation: 43

Cannot Connect from remote client to kafka server on digital ocean

I'm trying to test the connectivity from one environment, in to a digital ocean drop running our kafka server.

I'm attempting to run ./kafka-console-producer.sh --broker-list <HOST_NAME>:9092 --topic <topic>

And I'm consistently getting 

[2017-10-17 14:38:59,438] WARN Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) [2017-10-17 14:38:59,490] WARN Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) [2017-10-17 14:38:59,542] WARN Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) [2017-10-17 14:38:59,644] WARN Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) [2017-10-17 14:38:59,696] WARN Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

Steps taken so far:

Relevant section of my server.properties file (HOST_NAME is an alias for the actual host name to preserve privacy): 

listeners=PLAINTEXT://HOST_NAME:9092
host.name=HOST_NAME
# Hostname and port the broker will advertise to producers and consumers. If not set, 
# it uses the value for "listeners" if configured.  Otherwise, it will use the value
# returned from java.net.InetAddress.getCanonicalHostName().
advertised.listeners=PLAINTEXT://HOST_NAME:9092
advertised.host.name=HOST_NAME
advertised.port=9092

I can connect from the localmachine to itself, but I'm at the end of my rope for how to get a dumb little test message to come across the wire.

What am I doing wrong?

Upvotes: 0

Views: 4577

Answers (1)

DoctaMag
DoctaMag

Reputation: 43

This was solved by modifying the iptables config on my remote host by adding this rule to my iptables config:

-A INPUT -s [hosts went here] -p tcp -m state --state NEW -m tcp --dport 9092 -j ACCEPT

I had to allow TCP connections to port 9092 on the remote host. Our infrastructure team blocks all but specific ports in our development environment which was blocking dev from using telnet.

To summarize: Open the ports on your remote kafka server, with an iptable configuration. Confirm your localhost can send tcp connection data to that port.

Upvotes: 2

Related Questions