CODEWITHSUNDEEP
google-cloud-dataflowapache-beam
jimmy
jimmy

Reputation: 4881

Apache Beam Dataflow Jobs started failing with: Workflow failed

I've been running batch jobs for over a week now with DataflowRunner without a problem but all of a sudden starting from today the jobs started failing with the error message below. The workers don't seem to start and there's no log in stackdriver at all.

Anything I'm missing here?

Dataflow SDK version: 2.0.0
Submitted job: 2017-08-29_09_43_20-9537473353894635176
2017-08-29 16:44:24 ERROR MonitoringUtil$LoggingHandler:101 - 2017-08-29T16:44:22.277Z: (54a5da9d57fd266d): Workflow failed.

EDIT: If I remove --zone=europe-west2-b from the batch run it works which indicates that there might be something wrong with this zone.

Upvotes: 0

Views: 672

Answers (1)

Lara Schmidt
Lara Schmidt

Reputation: 309

I took a look at your job. It failed because it couldn't get quota to bring up the workers. Likely you do not have quota in that zone. This error is not handled back correctly, but it should be fixed in the next release.

Upvotes: 1

Related Questions