CODEWITHSUNDEEP
amazon-ec2apache-kafkaapache-zookeeperhyperledger-fabric
mohammad obaid
mohammad obaid

Reputation: 413

Kafka container timeout

I have deployed hyperldger-fabric kafka based ordering service using ansible on aws. Everything working fine for me till yesterday. Today when I launch a network , kafka container unable to communicate with zookeeper. Here are docker logs of kafka containers

[2017-11-16 08:23:36,075] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1223)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:155)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:129)
    at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:89)
    at kafka.utils.ZkUtils$.apply(ZkUtils.scala:71)
    at kafka.server.KafkaServer.initZk(KafkaServer.scala:278)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:168)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
    at kafka.Kafka$.main(Kafka.scala:67)
    at kafka.Kafka.main(Kafka.scala)
[2017-11-16 08:23:36,077] INFO shutting down (kafka.server.KafkaServer)
[2017-11-16 08:23:36,080] INFO shut down completed (kafka.server.KafkaServer)
[2017-11-16 08:23:36,081] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1223)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:155)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:129)
    at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:89)
    at kafka.utils.ZkUtils$.apply(ZkUtils.scala:71)
    at kafka.server.KafkaServer.initZk(KafkaServer.scala:278)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:168)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
    at kafka.Kafka$.main(Kafka.scala:67)
    at kafka.Kafka.main(Kafka.scala)
[2017-11-16 08:23:36,082] INFO shutting down (kafka.server.KafkaServer)

I havent change any code or anything else that's why I am unable to figure out what causes the problem . Any trick to solve this issue?

Upvotes: 0

Views: 440

Answers (1)

mohammad obaid
mohammad obaid

Reputation: 413

Finally fixed that issue. It was due to iptables setting which blocks icmp packets to be forwarded from flannel interface to docker interface thus docker containers couldn't communicate to each other. By adding iptable rules everything works fine for me .

Upvotes: 1

Related Questions