CODEWITHSUNDEEP
dockerapache-kafkadocker-compose
André Ferrari Moukarzel
André Ferrari Moukarzel

Reputation: 62

How can I pipe Kafka messages into a docker container?

I have a computer currently hosting the zookeeper and kafka servers.

I also have, in the same machine, a script that consumes messages sent to the local kafka server. The consumer script works as intended if I run it directly.

I want to run the consumer script from inside a docker container.

I have successfully built and run a container that runs the consumer script, but it waits forever for the kafka messages.

How can I make the kafka messages be redirected into the container? Is the only way to do this to host the zookeeper and kafka servers directly in the container?

Upvotes: 0

Views: 752

Answers (1)

Arthur Erlendsson
Arthur Erlendsson

Reputation: 176

By default, the consumer script's container is isolated from the host networking stack. The Kafka consumer needs to be able to see your brokers and Zookeeper instances running on your host machine.

There are number of solutions to this issue discussed here: Forward host port to docker container

A simple short-term solution is running your container on host networking by passing in --network=host, allowing the consumer container to share namespace with the host (e.g. you can use 'localhost:9092'). Note that this only works on Linux hosts.

Docker docs on using host networking: https://docs.docker.com/network/host/

Upvotes: 1

Related Questions