CODEWITHSUNDEEP
google-cloud-dataflowapache-beamapache-beam-io
Billy Jacobson
Billy Jacobson

Reputation: 1703

Apache Beam that uses time as an input

I'm looking to create a Beam input that executes every second and just outputs the time as an input. I know I can do a pcollection that is from numbers like this 

p.apply(Create.of(1, 2, 3, 4, 5))
        .setCoder(VarIntCoder.of())

and I could just create a really large array of numbers and window them to be every second, but is there a better way to do this? Thanks.

Upvotes: 0

Views: 58

Answers (1)

Billy Jacobson
Billy Jacobson

Reputation: 1703

Figured out that this can be done with GenerateSequence for bounded or unbounded sets. In order to get 1 data point out per second, I can use the withRate function and if I don't include a "to" then my pcollection will be unbounded.

p.apply(GenerateSequence.from(0).withRate(1, new Duration(1000)))

Upvotes: 2

Related Questions