CODEWITHSUNDEEP
javascriptapache-kafkakafka-consumer-apiapache-kafka-connectkafka-producer-api
JAN
JAN

Reputation: 21855

Kafka.JS refuses to connect <<[BrokerPool] Failed to connect to seed broker, trying another broker from the list>>

Consider the Kafka Producer :

const { Kafka, logLevel } = require('kafkajs');

const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['kafka1:9092', 'kafka2:9092'],
  logLevel: logLevel.ERROR,
});

const run = async () => {
  const producer = kafka.producer();

  await producer.connect();
  await producer.send({
    topic: 'test-topic',
    messages: [{ value: 'Hello KafkaJS user!' }],
  });

  await producer.disconnect();
};

run();

Whenever I hit : node producer.js , the response is :

C:\Development-T410\Kafka>node producer
{"level":"ERROR","timestamp":"2020-10-09T08:24:07.646Z","logger":"kafkajs","message":"[Connection] Connection timeout","broker":"kafka1:9092","clientId":"my-app"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:07.674Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection timeout","retryCount":0,"retryTime":325}
{"level":"ERROR","timestamp":"2020-10-09T08:24:09.004Z","logger":"kafkajs","message":"[Connection] Connection timeout","broker":"kafka2:9092","clientId":"my-app"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:09.006Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection timeout","retryCount":1,"retryTime":630}
{"level":"ERROR","timestamp":"2020-10-09T08:24:10.639Z","logger":"kafkajs","message":"[Connection] Connection timeout","broker":"kafka1:9092","clientId":"my-app"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:10.640Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection timeout","retryCount":2,"retryTime":1228}
{"level":"ERROR","timestamp":"2020-10-09T08:24:12.871Z","logger":"kafkajs","message":"[Connection] Connection timeout","broker":"kafka2:9092","clientId":"my-app"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:12.872Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection timeout","retryCount":3,"retryTime":2104}
{"level":"ERROR","timestamp":"2020-10-09T08:24:13.464Z","logger":"kafkajs","message":"[Connection] Connection error: getaddrinfo ENOTFOUND kafka1","broker":"kafka1:9092","clientId":"my-app","stack":"Error: getaddrinfo ENOTFOUND kafka1\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:14.789Z","logger":"kafkajs","message":"[Connection] Connection error: getaddrinfo ENOTFOUND kafka2","broker":"kafka2:9092","clientId":"my-app","stack":"Error: getaddrinfo ENOTFOUND kafka2\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:15.722Z","logger":"kafkajs","message":"[Connection] Connection error: getaddrinfo ENOTFOUND kafka1","broker":"kafka1:9092","clientId":"my-app","stack":"Error: getaddrinfo ENOTFOUND kafka1\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:15.978Z","logger":"kafkajs","message":"[Connection] Connection timeout","broker":"kafka1:9092","clientId":"my-app"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:15.979Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection timeout","retryCount":4,"retryTime":4610}
{"level":"ERROR","timestamp":"2020-10-09T08:24:17.059Z","logger":"kafkajs","message":"[Connection] Connection error: getaddrinfo ENOTFOUND kafka2","broker":"kafka2:9092","clientId":"my-app","stack":"Error: getaddrinfo ENOTFOUND kafka2\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:17.980Z","logger":"kafkajs","message":"[Connection] Connection error: getaddrinfo ENOTFOUND kafka1","broker":"kafka1:9092","clientId":"my-app","stack":"Error: getaddrinfo ENOTFOUND kafka1\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:21.593Z","logger":"kafkajs","message":"[Connection] Connection timeout","broker":"kafka2:9092","clientId":"my-app"}
{"level":"ERROR","timestamp":"2020-10-09T08:24:21.594Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection timeout","retryCount":5,"retryTime":8310}
(node:9624) UnhandledPromiseRejectionWarning: KafkaJSNonRetriableError
  Caused by: KafkaJSConnectionError: Connection timeout
    at Timeout.onTimeout [as _onTimeout] (C:\Development-T410\Kafka\node_modules\kafkajs\src\network\connection.js:165:23)
    at listOnTimeout (internal/timers.js:549:17)
    at processTimers (internal/timers.js:492:7)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:9624) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:9624) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
{"level":"ERROR","timestamp":"2020-10-09T08:24:22.857Z","logger":"kafkajs","message":"[Connection] Connection error: getaddrinfo ENOTFOUND kafka2","broker":"kafka2:9092","clientId":"my-app","stack":"Error: getaddrinfo ENOTFOUND kafka2\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26)"}

Any idea how to solve this ?

Upvotes: 8

Views: 27603

Answers (5)

northernwind
northernwind

Reputation: 637

I has the same issue with a similar setup (except kafka placed in docker container). In my case the issue was in corporate VPN connected (cisco any connect). I just turned it off.

Maybe it will be helpfull for someone.

Upvotes: 1

Maximiliano De Lorenzo
Maximiliano De Lorenzo

Reputation: 1024

I had a similar issue, the CLI was working well but kafkajs was failing.

In my case I was trying to use localhost:9092 as broker, but as I was using docker-compose I had to use kafka:9092 (because my service was called kafka 🤦).

const kafka = new Kafka({ 
    clientId: 'my-app',
    brokers: ['kafka:9092'],
})

Upvotes: 0

Harika
Harika

Reputation: 1

This is because , when your broker-list got changed, but on the application side you trying to get the old broker list itself. You can either restart your pods so that they can retrive new broker list and start processing the messages.

Upvotes: 0

karan shah
karan shah

Reputation: 370

The error message suggests that the brokers are not reachable and hence your nodejs code is not able to establish a connection. Check

  • Kafka brokers and Zookeeper are up and running.
  • Double check the ports on which kafka is running.
  • The DNS/Hostname you are using is resolvable.

You can use cli to first ensure the setup is correct and run a producer & consumer.

Going through this tutorial might help.

Upvotes: 0

Ravi
Ravi

Reputation: 108

I was trying to reproduce the same issue. But this code is working fine with brokers: ['localhost:9092']. There is no issue with this code you should check your Kafka server. There might be a possibility that your Kafka server is not configured properly.

Try creating a topic from the terminal. You can find all the scripts inside your Kafka installation folder's bin directory.

Run

kafka-topics.sh --bootstrap-server localhost:9092 --create --topic test-topic --partitions 1 --replication-factor 1

Check if the topic is created successfully. If not there is a problem with your Kafka broker.

Upvotes: 2

Related Questions