CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-dataflowdataflow
farhawa
farhawa

Reputation: 10407

Dataflow Flex template job is Queued

I am trying to reproduce this tutorial to run a Flex Template on Dataflow.

When I submit the job, I can see it in the console but it's not started and marked as Queued. Does this mean that the job is submitted in a FlexRS mode ? How can I start directly the job after submitting it ?

Upvotes: 4

Views: 3297

Answers (2)

Travis Webb
Travis Webb

Reputation: 15018

This looks like a bug, where an upstream dependency (pyarrow?) started requiring git in order to build, but the base image does not currently include git. I have filed an issue here: https://issuetracker.google.com/issues/176570473

Upvotes: 1

danielm
danielm

Reputation: 3010

The "Queued" status for Flex Template jobs means that your container is running on a VM to build the pipeline and start the job. If the job stays in Queued for more than a few minutes, that indicates the this process got stuck. You can view the logs for this VM in the Dataflow UI in the "Job Logs" section.

Upvotes: 5

Related Questions