CODEWITHSUNDEEP
apache-kafkaapache-zookeeper
jumping_monkey
jumping_monkey

Reputation: 7789

Testing "fail-over" on Kafka

Set-up 1:

OS: Windows 10

ZooKeeper

3 ZooKeeper instances downloaded from Apache(tested with v3.5.6 and v.3.4.14):
(1) apache-zookeeper-3.5.6-bin_1
(2) apache-zookeeper-3.5.6-bin_2 (Copy of 1)
(3) apache-zookeeper-3.5.6-bin_3 (Copy of 1)

zoo.cfg:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper_3.4.14_1
clientPort=2181
admin.serverPort=10081
server.1=localhost:2881:3881
server.2=localhost:2882:3882
server.3=localhost:2883:3883
4lw.commands.whitelist=*

zoo.cfg:
...
dataDir=/tmp/zookeeper_3.4.14_2
clientPort=2182
admin.serverPort=10082
...

zoo.cfg:
...
dataDir=/tmp/zookeeper_3.4.14_3
clientPort=2183
admin.serverPort=10083
...

myid file in dataDir with values 1,2 and 3 respectively

Kafka

2 Kafka instances:
(1) kafka_2.12-2.3.0_1
(2) kafka_2.12-2.3.0_2 (Copy of 1)

server.properties:
...
broker.id=1
listeners=PLAINTEXT://:9091
log.dirs=/tmp/kafka-logs-1
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183
...

server.properties:
...
broker.id=2
listeners=PLAINTEXT://:9092
log.dirs=/tmp/kafka-logs-2
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183
...

Spring

spring-boot-starter-* 2.2.0.RELEASE
spring-kafka-2.3.1.RELEASE

=====================================================================

Set-up 2:

Same as set-up 1, the only difference being that instead of using the ZooKeeper downloaded from Apache, i am using the ZooKeeper that comes with Kafka.

=====================================================================

Issue

The issue is that when i bring 1 Kafka down:

=> Set-up 1 will not fail-over, meaning that when i produce a message, the Kafka that is up is not receiving the message => Set-up 2 will fail-over, meaning that when i produce a message, the Kafka that is up will receive the message

Do you guys see anything wrong with Set-up 1?

P.S If you need more details, i am happy to provide.

Upvotes: 0

Views: 342

Answers (1)

jumping_monkey
jumping_monkey

Reputation: 7789

In case it helps someone:

I had 2 Kafka instances, but my replication-factor for any topic created was 1(due to my misunderstanding/misinterpretation of its meaning).

What this means is that, at the time of topic creation, the topic/s will be either created in Kafka-1 or Kafka-2, not both. As such, when i tried to do a fail-over, the fail-over will fail, depending on which topic i am writing to, and which Kafka was brought down.

In short, if you have X Kafka instances, topic replication-factor needs to be X (Same goes for offsets.topic.replication.factor)

Upvotes: 1

Related Questions