CODEWITHSUNDEEP
apache-kafkaapache-kafka-streamsapache-kafka-connectksqldb
alphanumeric
alphanumeric

Reputation: 19329

How to read data from Kafka Topic with Stream

I would like to create MY_STREAM in Kafka that reads the data messages from MY_TOPIC and push it to another TARGET_TOPIC. Using KSQL statement:

CREATE STREAM MY_STREAM WITH (KAFKA_TOPIC='TARGET_TOPIC') AS SELECT * FROM MY_TOPIC;

results to an error MY_TOPIC doesn't exist . Apparently Stream cannot be created with read access to MY_TOPIC. Stream can only read from another Stream. Is there a way to make MY_STREAM to read data from MY_TOPIC?

P.S. Appended later:

Regardless on how Stream was created, with "as select" or not it is created with Topic pushing and pulling data from Stream.

Upvotes: 0

Views: 1158

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191681

Perhaps review the 101 Course videos.

You can PRINT a topic to do a basic consumer, but in order to SELECT, you need to CREATE a STREAM over the topic.

For example, CREATE STREAM MY_STREAM (fields...) WITH (kafka_topic='MY_TOPIC', value_format="...")

Also, it is redundant to do CREATE STREAM ... AS SELECT * without any filter/grouping/calculations since those would be the exact same stream. This use-case should be limited to type-conversions and re-partitioning of topics, which doesn't sound like you're doing.

Upvotes: 1

Related Questions