CODEWITHSUNDEEP
apache-kafkaksqldb
Rahul
Rahul

Reputation: 469

How can i run KSQL stream with compression

I am using KSQL for processing Kafka data. I can able to create persistent kafka streams, but the output topic for this stream is not compressed one. How can i define the compression type while creating the steream.

CREATE STREAM TARGET_TOPIC WITH **(COMPRESSION='SNAPPY')** AS SELECT * FROM SOURCE PARTITION BY ID

I am expecting giving compression configuration like above but the above syntax is not correct. If compression is possible in ksql please assist on this.

Thanks

Upvotes: 1

Views: 1013

Answers (1)

Andrew Coates
Andrew Coates

Reputation: 1893

There are two ways to achieve this.

  1. You can use the SET command in KSQL to set the producer config that controls compression before creating the stream, e.g. 
SET 'compression.type'='snappy';
CREATE STREAM TARGET_TOPIC WITH **(COMPRESSION='SNAPPY')** AS SELECT * FROM SOURCE PARTITION BY ID
  1. or you can set it in the ksql-server.properties file if you want ALL output to be compressed.

Upvotes: 2

Related Questions