CODEWITHSUNDEEP
javaapache-kafka
mGm
mGm

Reputation: 262

Create Multiple streams with state store to return latest values per key

I am consuming an inputTopic and from which I am extracting a list of String (topic names).

List<String> Topics = {"Topic1", "Topic2", "Topic3"}

I need to create KStream for each topic alongwith their state stores which should contains only last value for each key.

These topic (i.e, topic1, topic2, and topic3 are already exists and contains above millions of messages in them).

How to create these (multi) streams that consumes all messages and from each topic asynchronously and write them to an outputTopic.

I am new to kafka environment.

Upvotes: 0

Views: 30

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191681

A KStream can consume multiple topics at once, but a KTable doesn't store data by a topic name. You'd need to filter per topic name, such as with branch operator or TopicNameExtractor, or just create 3 unique KStream instances, then create unique statestores with toTable

KTables automatically store values with latest keys

Upvotes: 0

Related Questions