CODEWITHSUNDEEP
apache-kafka
Ris90
Ris90

Reputation: 841

What kafka.common.OffsetOutOfRangeException means

I'm trying to load data through Apache Kafka and constantly getting this error:

kafka.common.OffsetOutOfRangeException: offset 1003786 is out of range at kafka.log.Log$.findRange(Log.scala:46) at kafka.log.Log.read(Log.scala:264) at kafka.server.KafkaRequestHandlers.kafka$server$KafkaRequestHandlers$$readMessageSet(KafkaRequestHandlers.scala:112) at kafka.server.KafkaRequestHandlers$$anonfun$2.apply(KafkaRequestHandlers.scala:101) at kafka.server.KafkaRequestHandlers$$anonfun$2.apply(KafkaRequestHandlers.scala:100) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34) at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34) at scala.collection.TraversableLike$class.map(TraversableLike.scala:206) at scala.collection.mutable.ArrayOps.map(ArrayOps.scala:34) at kafka.server.KafkaRequestHandlers.handleMultiFetchRequest(KafkaRequestHandlers.scala:100) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$3.apply(KafkaRequestHandlers.scala:40) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$3.apply(KafkaRequestHandlers.scala:40) at kafka.network.Processor.handle(SocketServer.scala:296) at kafka.network.Processor.read(SocketServer.scala:319) at kafka.network.Processor.run(SocketServer.scala:214) at java.lang.Thread.run(Thread.java:724)

What does this exception mean and how I can fix it?

Upvotes: 9

Views: 12373

Answers (1)

user2720864
user2720864

Reputation: 8171

OffsetOutOfRangeException generally indicates that client has requested a range no longer available on the server.
This could happen as the topic log does not exists anymore based on the retention policy in your Kafka setup.
if you are using SimpleConsumer you will need to handle the OffsetOutOfRange Exception in your code. Ideally your consumer should issue an OffsetRequest to get the latest/earliest offset currently available on the server, and then use this in your FetchRequest (as a parameter)

Upvotes: 13

Related Questions