CODEWITHSUNDEEP
javaapache-kafkaaws-msk
Kapil Devmurari
Kapil Devmurari

Reputation: 159

AWS MSK gives warning

we are working on spring boot application and we have implemented MSK AWS (Kafka) , producer and consumer communication is working as expected but once I checked logs it is giving warning as below.

Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available

Spring boot yml configuration file

spring:
   cloud:
     config:
       enabled: true
   application:
      name: my-service
   kafka:
     producer:
       bootstrap-servers: broker1MSKUrl:9092,broker2Url:9092
     consumer:
       bootstrap-servers: broker1MSKUrl:9092,broker2Url:9092

Spring boot kafka consumer class

@Value(value = "${spring.kafka.consumer.bootstrap-servers}")
private String bootstrapAddress;

    Map<String, Object> props = new HashMap<>();
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, 
    bootstrapAddress);
    props.put(ConsumerConfig.GROUP_ID_CONFIG, groupId);
    props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, 
    bootstrapAddress);
    props.put(JsonDeserializer.TRUSTED_PACKAGES, "*");
    AdminClient adminClient = AdminClient.create(props);
    ListTopicsOptions listTopicsOptions = new ListTopicsOptions();
    listTopicsOptions.listInternal(true);

we are wondering why we are getting localhost warning while we have connected to AWS MSK

This is our MSK configuration

auto.create.topics.enable=true
default.replication.factor=3
min.insync.replicas=2
num.io.threads=8
num.network.threads=5
num.partitions=1
num.replica.fetchers=2
replica.lag.time.max.ms=30000
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
socket.send.buffer.bytes=102400
unclean.leader.election.enable=true
zookeeper.session.timeout.ms=18000

Upvotes: 0

Views: 635

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191973

You're creating an AdminClient. So you need to set spring.kafka.admin.bootstrap-servers, not just the consumer/producer. Otherwise, the admin client will default to localhost.

Or use spring.kafka.bootstrap-servers to see all three client types.

Also, admin client doesn't use GROUP_ID_CONFIG or TRUSTED_PACKAGES... Try to limit what properties you set to specific clients. For example, bootstrap servers can come from CommonClientConfigs class, but that's also not necessary as Spring Boot will auto-wire the property mentioned above.

Upvotes: 1

Related Questions