CODEWITHSUNDEEP
apache-beamparquetapache-beam-io
manoveg
manoveg

Reputation: 483

How to write a partitioned parquet file in apache beam java

I am new to Apache Beam and not sure how to accomplish this task. I want to write a partitioned parquet file using Apache Beam in Java. Data is read from Kafka and I want the file to have a new partition every hour. The timestamp columns is present in the data.

Upvotes: 0

Views: 746

Answers (1)

Alexey Romanenko
Alexey Romanenko

Reputation: 1443

Try to use FixedWindows for that. There is an example of windowed WordCount that writes every window into separate text file - so, I believe, it can be adapted for your case.

Upvotes: 0

Related Questions