CODEWITHSUNDEEP
apache-kafkakafka-producer-api
Suraj
Suraj

Reputation: 75

Do all Kafka broker IPs have to be specified when producing a message

I have 3 broker kafka cluster with 3 zookeeper.

My question is if we have to give only one IP address in producer-console.sh file like below

/kafka-console-producer.sh --broker-list 192.168.7.110:9092 --topic test

or all the three ip addressess

./kafka-console-producer.sh --broker-list 192.168.7.110:9092,192.168.5.110:9092,192.168.3.111:9092 --topic test

What will happen if I provide only one IP to produce messages and that IP is shutdown after sometime. Will I be able to produce messages through that IP or not or I have to give all the IP addresses?

Upvotes: 3

Views: 2336

Answers (1)

Robin Moffatt
Robin Moffatt

Reputation: 32080

Check out the Producer config docs, which describe the purpose of bootstrap.servers (bootstrap-servers / broker-list are synonyms):

A list of host/port pairs to use for establishing the initial connection to the Kafka cluster.

The client will make use of all servers irrespective of which servers are specified here for bootstrapping—this list only impacts the initial hosts used to discover the full set of servers. This list should be in the form host1:port1,host2:port2,....

Since these servers are just used for the initial connection to discover the full cluster membership (which may change dynamically), this list need not contain the full set of servers (you may want more than one, though, in case a server is down).

So if you only provide one IP, and that IP is then shut down, your producer will subsequently fail when it tries to connect. But you could, for example, supply two IPs, so that if one fails the producer can still connect to a different one. But the broker to which the actual messages are sent is not impacted by this.

See also this answer here.

Upvotes: 3

Related Questions