CODEWITHSUNDEEP
javaamazon-web-servicesamazon-ec2spring-kafka
jojupiter
jojupiter

Reputation: 68

Impossible to create a topic on KAFKA

I try to create a topic on apache KAFKA automatically with spring boot JAVA. For that I followed this tutorial: https://www.baeldung.com/spring-kafka But I manage to connect to the server, but I can't create a topic. The KAFKA server is hosted on an EC2 AWS instance.

Here is a part of the error obtained in the Intellij console

022-05-08 10:21:22.889  WARN 20424 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient   : [AdminClient clientId=adminclient-1] Error connecting to node ip-172-31-XX-XX.eu-west-2.compute.internal:9092 (id: 0 rack: null)

java.net.UnknownHostException: ip-172-31-XX.XX.eu-west-2.compute.internal
    at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:797) ~[na:na]
    at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505) ~[na:na]
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364) ~[na:na]
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298) ~[na:na]
    at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:511) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:468) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:979) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.sendEligibleCalls(KafkaAdminClient.java:1127) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1387) ~[kafka-clients-3.0.1.jar:na]
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1330) ~[kafka-clients-3.0.1.jar:na]
    at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

image capture from offset explorer

Here my code :

@Configuration
public class KafkaTopicConfig {
    @Value(value = "${kafka.bootstrapAddress}")
    private String bootstrapAddress;

    @Bean
    public KafkaAdmin kafkaAdmin() {
        Map<String, Object> configs = new HashMap<>();
        configs.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress);
        return new KafkaAdmin(configs);
    }

    @Bean
    public NewTopic topic1() {
        return new NewTopic("baeldung", 1, (short) 1);
    }
}

Upvotes: 0

Views: 796

Answers (1)

jojupiter
jojupiter

Reputation: 68

I found the solution, I had not properly configured the config/server.properties file of Apache KAFKA. Uncomment the following lines: listeners=PLAINTEXT://:9092 advertised.listeners=PLAINTEXT://your.host.name:9092

replace "you.host.name" with the "public ip address" of the EC2 server where Apache KAFKA is installed. Open port 9092.

listeners is what the broker will use to create the server sockets.

advertised.listeners is what the clients will use to connect to the brokers.

Upvotes: 1

Related Questions