CODEWITHSUNDEEP
scalanullpointerexceptionapache-kafkakafka-consumer-api
achuz_dev
achuz_dev

Reputation: 1

Kafka consumer in Scala

I am trying to write a Kafka consumer code using Scala.

I have a Kafka cluster (server A) with some messages already posted to the topic (test-topic) and I am consuming it from another server (say B) which has a connection established from server B to A. (I am using spark-shell to code).

Below is the snippet I am trying to execute in server B, but I am getting a null pointer exception while executing the code.

import java.util._
import java.util.Properties
import org.apache.kafka.clients.consumer.KafkaConsumer
import org.apache.kafka.clients.consumer.ConsumerConfig
import scala.collection.JavaConverters._
import org.apache.kafka.common.TopicPartition
val  props = new Properties()
props.put("group.id","group-test")
props.put("auto.commit.interval.ms","1000")
props.put("auto.offset.reset","earliest")
props.put("bootstrap.servers", "<kafka server ip>:9092")
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
props.put("partition.assignment.strategy","org.apache.kafka.clients.consumer.RangeAssignor")
val consumer = new KafkaConsumer[String, String](props)
consumer.subscribe("test-topic")
 while(true){
   val records=consumer.poll(100)
   println(consumer)
   println(records)
   for (record<-records.asScala){
    println(record)
   }
 }



Error --
org.apache.kafka.clients.consumer.KafkaConsumer@29451e22
null
java.lang.NullPointerException
  at scala.collection.convert.Wrappers$JMapWrapperLike$$anon$2.<init>(Wrappers.scala:275)
  at scala.collection.convert.Wrappers$JMapWrapperLike$class.iterator(Wrappers.scala:274)
  at scala.collection.convert.Wrappers$JMapWrapper.iterator(Wrappers.scala:292)
  at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
  at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
  ... 59 elided

Please help to resolve the issue.

Upvotes: 0

Views: 1988

Answers (1)

ziad
ziad

Reputation: 404

consumer.poll(100) is returning null.

Increase the timeout and test if records != null before looping them.

Upvotes: 2

Related Questions