CODEWITHSUNDEEP
apache-kafkakafka-producer-api
SourceSimian
SourceSimian

Reputation: 730

Is it possible to produce to a kafka topic when only 1 of the brokers is reachable?

Is it possible to produce to a Kafka topic when only 1 of the brokers is reachable from the producer, none of the zookeeper nodes are reachable from the producer, but all of the brokers are healthy and are reachable from each other?

For example, this would be required if I were to produce messages via an SSH tunnel. If this were for a temporary push I could possibly create the topic with replication factor 1 and have all partitions assigned to the broker in question, and reassign the partitions after the fact, but I'm hoping there is a more flexible setup.

This is all using the java client.

Upvotes: 2

Views: 1755

Answers (1)

Mickael Maison
Mickael Maison

Reputation: 26875

Producers don't interact with Zookeeper so it's not an issue.

The only requirement for Producers is to be able to connect to the brokers that are leaders for the partitions they want to use.

If the broker you connect to is the leader for the partitions you want to use, then yes you can produce to it.

Otherwise it's not going to work. Also creating a topic may not help as its partitions could be assigned to any brokers. Also in order to create a topic, a client has to connect to the controller which may not be the broker you can reach.

If you can only connect to 1 "thing", you may want to consider using something like a REST Proxy. Your "isolated" environment could send REST requests to the proxy which is able to connect to all brokers in the cluster.

Upvotes: 4

Related Questions