CODEWITHSUNDEEP
apache-kafkaksqldb
devesh joshi
devesh joshi

Reputation: 155

ksql get distinct values in a column

How do I achieve in ksql

select distinct(`columnName`) from stream_name;

without using a table? I don't want to have a table compacted topic in my broker.

Upvotes: 0

Views: 196

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191681

You have an endless stream. An aggregation is needed. Without a table, events will be silently ignored, e.g

COLLECT_SET(col1) => ARRAY
Returns an array containing the distinct values of col1 from each input row (for the specified grouping and time window, if any).

However,

The size of the result ARRAY can be limited to a maximum of ksql.functions.collect_set.limit entries, and any values beyond this limit are ignored silently.

don't want to have a table compacted topic in my broker

You already have at least one, by default - __consumer_offsets

Upvotes: 1

Related Questions