CODEWITHSUNDEEP
apache-crunch
Vivek Rai
Vivek Rai

Reputation: 73

How could I define the DoFn in apache crunch having "void" data type?

Basically, I don't need output from DoFn, just want to update some mysql db for each record I am getting in DoFn. So how could I define DoFn having void data type ? Basically I don't want to emit anything form DoFn.

Upvotes: 1

Views: 294

Answers (1)

hlagos
hlagos

Reputation: 7947

it is not possible. Once Crunch generate the graph for the execution, it determines if there is or not an output to be written or data to be materialized (pretty similar to a action). Even to accomplish what you want, you will need to do a workaround like create a DoFn<T,String> are code for write or materialize the PCollection result from your DoFn, even if you already know that there is not going to be any outputs.

Upvotes: 0

Related Questions