CODEWITHSUNDEEP
apache-flinkflink-batch
Saravanan
Saravanan

Reputation: 19

Apache Flink - Dataset api - Side outputs

Does Flink supports Side Outputs feature in Dataset(Batch Api) ? If not, how to handle valid and invalid records when loading from file ?

Upvotes: 1

Views: 217

Answers (1)

David Anderson
David Anderson

Reputation: 43707

You can always do something like this:

DataSet<EventOrInvalidRecord> goodAndBadTogether = input.map(new CreateObjectIfPossible())
goodAndBadTogether.filter(new KeepOnlyGood())...
goodAndBadTogether.filter(new KeepOnlyBad())...

Another reasonable option in some cases is to go ahead and use the DataStream API, even if you don't have streaming sources.

Upvotes: 3

Related Questions