Gcloud Dataflow Step Execution Time

Question

I'm using gcloud dataflow job and want individual execution times for all the steps in my dataflow including nested transforms. I'm using a streaming dataflow and the pipeline currently looks like this:

Current dataflow

Can anyone please suggest a solution?

Answer 1

Answer is WallTime. You can access this info by clicking one of the task in your pipeline(even nested). Elapsed time of a job is the total time takes to complete your dataflow job while wall time is the sum time taken to run each step by the assigned workers. See the below image for more details.