CODEWITHSUNDEEP
apache-kafkaapache-kafka-streams
eomercelik
eomercelik

Reputation: 595

Why should I use KStream or KTable?

I read but I couldn't understand too much. I read that I can use KTable instead of log compaction. Or it has many more features. However, I could not find a good example in this regard. I also couldn't see it in a nice source explaining the logic of work. Can you explain a good example of ktable and kstream and what can I do?

Upvotes: 7

Views: 10778

Answers (3)

OneCricketeer
OneCricketeer

Reputation: 191874

I read that I can use KTable instead of log compaction

A KTable depends on a compacted state store topic. They are not exclusive options

Or it has many more features

Well, why would you use a cache or Hashmap? The same answer can be applied to a KTable. The extra feature is that it can be shared and distributed across multiple instances of your application

You can do more research on the "Stream-Table Duality".

example of ktable and kstream and what can I do?

A KStream is an audit log of all, or a filtered subset of events in the topic. It's hard to quickly pick out any given event

A KTable holds the most recent keyed event from a stream and allows for fast key lookups

A counter is the simplest example; You have a stream of events (say, words from hashtags)

#kafka is great
working with #kafka today
#streaming all the things

So, that's the stream. You then need to consume this stream into an aggregated table, parsing out and counting the hashtags, resulting in key value pairs

(kafka, 2)
(streaming, 1)

However, if you were to query the table immediately after the first event (before the table consumed the remaining events) you'd only see (kafka, 1).

Upvotes: 3

Anil
Anil

Reputation: 331

A KTable is an abstraction of a changelog stream, where each data record represents an update. More precisely, the value in a data record is interpreted as an “UPDATE” of the last value for the same record key,

Kafka Streams is a library for building streaming applications, specifically applications that transform input Kafka topics into output Kafka topics.

KStream handles the stream of records. On the other hand, KTable manages the changelog stream with the latest state of a given key. Each data record represents an update in KTable.

KStreams are stateless whereas KTable is stateful.

Upvotes: 5

Wajih
Wajih

Reputation: 93

As per my understanding, we should use KStream for the data which we want to receive from the source and are okay with data repetitions.

We should use KTable when we do not want to keep the repetitions of the records and we are only interested in the latest value for the respective key.

Kstream is continuous.

Upvotes: 4

Related Questions