CODEWITHSUNDEEP
pythonapache-kafkavagrant
Manuel Simões
Manuel Simões

Reputation: 123

Error Apache Kafka ubuntu vagrant - advertised.listeners and listeners

I'm running a vagrant host machine Ubuntu 20.04 on 192.168.50.4:

config.vm.box = "bento/ubuntu-20.04"
...
config.vm.network "private_network", ip: "192.168.50.4"

Nothing too complex network wise, besides the LAN defaults.

I then installed Kafka following this steps (I know there's a Vagrant Box for it, I just wanted to install myself).

Now the problem(s):

a) - If i keep the defaults in server.properties:

#listeners=PLAINTEXT://:9092
...
#advertised.listeners=PLAINTEXT://your.host.name:9092

(both commented)

The producer and consumer sh files of the guest machine both work without any problem. But my python script on the host machine don't:

...
settings = {
    'bootstrap.servers': '192.168.50.4:9092',
    'group.id': 'mygroup',
    'client.id': 'client-1',
    'enable.auto.commit': True,
    'session.timeout.ms': 6000,
    'default.topic.config': {'auto.offset.reset': 'smallest'}
}
...

(Full example here)

Giving the error:

...
%3|1612854723.928|FAIL|client-1#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: vagrant.vm:9092: Failed to resolve 'vagrant.vm:9092': Name or service not known (after 29ms in state CONNECT)
...

I already tried:

listeners=PLAINTEXT://localhost:9092
...
advertised.listeners=PLAINTEXT://192.168.50.4:9092

This way the problem remains with:

%3|1612855333.958|FAIL|client-1#consumer-1| [thrd:192.168.50.4:9092/bootstrap]: 192.168.50.4:9092/bootstrap: Connect to ipv4#192.168.50.4:9092 failed: Connection refused (after 0ms in state CONNECT, 1 identical error(s) suppressed)

b) - If I just change listeners in server.properties to: 

...
listeners=PLAINTEXT://192.168.50.4:9092
...

The python script works from host machine but the producer and consumer sh files of the guest don't:

[2021-02-09 07:20:36,958] WARN [Producer clientId=console-producer] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)

Don't know any way to keep both working, connection from guest and from host. And it was not for lack of searching and analysis.

Upvotes: 0

Views: 329

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191874

Listeners is a bind address. Needs to be 0.0.0.0:(port) to open the server to all clients. Setting to localhost restricts to that machine, setting to an ip restricts to that interface/route

The simplest solution here is to advertise localhost:(port), then setup port forwarding in Vagrant from the advertised port to the port specified in the listeners config (these can be different ports)

Your first error is because the VM hostname is not known to your host unless you install some Vagrant plugin.

The more appropriate solution is to advertise the VM host-only IP address

Upvotes: 1

Related Questions