CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-dataflowapache-beam
Brice
Brice

Reputation: 366

Python Streaming pipeline on Google Dataflow missing some Element metrics

I am running a Python dataflow (beam 2.5) and was wondering why the step metrics are not showing after a beam.Map(lambda x: x.decode('utf-8')).

Everything is working fine, but the number of elements vanish. Step details for lambda

As you can see, the numbers from this point on are zero.

Is there something that I am missing?

Upvotes: 0

Views: 212

Answers (1)

Pablo
Pablo

Reputation: 11031

Support for streaming pipelines with Apache Beam Python SDK on Google Cloud Dataflow is in Beta.

Some features are not fully supported, and that's the case for some monitoring metrics. These will gradually be added as support is improved.

Upvotes: 1

Related Questions