CODEWITHSUNDEEP
javaapache-kafkaconfluent-platform
Eshwar P
Eshwar P

Reputation: 389

How to connect Kafka (Java) application from Windows to Confluent in Linux

I'm running Confluent 5.0 in Linux server using Winscp and Putty. I have Kafka (Java/Eclipse) application in Windows.

When I run Java application, its not identifying Kafka brokers in Confluent running on Linux.

I have tested my Java application which send data to Kafka topics in MACBook, by running Confluent 5.0 in MAC Terminal. Now I am trying to implement same Kafka application in Windows. Since Confluent is not supported in Windows, I'm running in Linux server.

I'm using Confluent instead of Apache Kafka because I'm using Schema-registry in my application.

By using netstat -tupln & curl -v http:/localhost:port no. figured out Kafka's running on 8082 and schema registry on 8081 details of ports. Below is my Kafka Properties in Java application.

public static Properties producerProperties() {

    // normal producer
    properties.setProperty("bootstrap.servers", "127.0.0.1:8082");
    properties.setProperty("acks", "all");
    properties.setProperty("retries", "10");
    // avro part
    properties.setProperty("key.serializer", StringSerializer .class.getName());
    properties.setProperty("value.serializer", KafkaAvroSerializer .class.getName());
    properties.setProperty("schema.registry.url", "http://127.0.0.1:8081");

    return properties;

}

public static Properties consumerProperties() {

   // Properties properties = new Properties();
    // normal consumer
    properties.setProperty("bootstrap.servers", "127.0.0.1:8082");
    //different for consumer
    properties.setProperty("group.id", "Avro-consumer");
    properties.setProperty("enable.auto.commit", "false");
    properties.setProperty("auto.offset.reset", "earliest");

    // avro part
    properties.setProperty("key.deserializer", StringDeserializer.class.getName());
    properties.setProperty("value.deserializer", KafkaAvroDeserializer.class.getName());
    properties.setProperty("schema.registry.url", "http://127.0.0.1:8081");
    properties.setProperty("specific.avro.reader", "true");

    return properties;
}

public static Properties streamsProperties() {

    // normal consumer
    properties.put(StreamsConfig.APPLICATION_ID_CONFIG, "com.github.ptn006");
    properties.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:8082");
    properties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");

    properties.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());
    properties.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());

    return properties;
}

Expected: Data written to Kafka topics.

Actual: WARN Connection to node -1 could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient:589)

Upvotes: 0

Views: 933

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191701

You need to make sure that advertised.listeners of the server.properties file in Kafka is resolvable by the Windows machine. Also ensure the firewall is allowing access (netstat -tupln | grep LIST), and look for your Kafka port listening on 0.0.0.0, for example.

Upvotes: 1

Related Questions