CODEWITHSUNDEEP
apache-kafkavirtualbox
mu234
mu234

Reputation: 63

How to communicate between a VM and host machine using kafka?

I have been trying to essentially use Apache-Kafka to communicate between an Arch producer and Ubuntu VM consumer run on virtualbox. I have been able to set up a communication between producer and consumer on a single machine by following the following youtube tutorial.

https://www.youtube.com/watch?v=VbSRS7kG5Cw

Essentially I use these 4 commands in 4 separate terminals. 

./bin/zookeeper-server-start.sh config/zookeeper.properties 

./bin/kafka-server-start.sh config/server.properties

./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic MyFirstTopic1

./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic MyFirstTopic1

I have tried setting up the zookeeper and kafka servers on both the VM and the host, and the producer on one and the consumer on the other, with the ip address in the consumer command changed from localhost to to the ip address of the producer. I have tried setting up the servers on the producer side, and keeping the rest the same.

Any help would be greatly appreciated

Upvotes: 1

Views: 1721

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191874

It's not clear which machine is which in the post, but you must configure the following properties, at a minimum, to listen on non-local addresses 

listeners=PLAINTEXT://0.0.0.0:9092 
advertised.listeners=PLAINTEXT://<external ip>:9092

Start with producing and consuming from localhost, then move on to do the same from a non-localhost machine, and then you can do either from different ones

Note: VMs typically don't have direct network access to their host machines, but if the broker is in a VM, you'll need to setup port forwarding as well

Upvotes: 1

Related Questions