How do I know how many VMs are set up to run my dataflow job?

Question

I use dataflow service to run my job to read data from GCS and write to BQ tables, jobs run successfully, I'd like to know how or where I can find info how many VMs google set up for my job?

thank you.

Answer 1

When running Cloud Dataflow pipelines using [Blocking]DataflowPipelineRunner, the default number of worker virtual machines is 3.

This value can be overridden on the command-line, using --numWorkers parameter.

All virtual machines, including those created by Cloud Dataflow, are visible in Google Developers Console, under section: Compute, Compute Engine, VM instances.