CODEWITHSUNDEEP
apache-kafkaconfluent-platformksqldb
Ritu
Ritu

Reputation: 21

How does KSQL read(consume msgs) data from a topic, already containing some messages

I have created a stream from Kafka using KSQL for a Kafka topic which already contains some messages. But not receiving any msg from that topic into stream created. Messages are in Avro format and generated after some interval.

I want to read from starting i.e. from earliest msg. Also tried setting offset property to earliest but not receiving any msg.

create stream sample_transition with(topic_name='transition',value_format='avro');

Upvotes: 1

Views: 2609

Answers (1)

Matthias J. Sax
Matthias J. Sax

Reputation: 62350

You need to set auto.offset.reset configuration correctly. Be default it's "latest". If you want new queried to read all data from a topic, you need to set it to "earliest".

ksql> SET 'auto.offset.reset'='earliest';

Compare: https://docs.confluent.io/current/ksql/docs/installation/server-config/config-reference.html#ksql-streams-auto-offset-reset

Upvotes: 1

Related Questions