CODEWITHSUNDEEP
google-cloud-dataflow
gcpman
gcpman

Reputation: 129

Dataflow job drain does not end for a long time

Draining of Dataflow streaming job does not end while 1 days.

No error log on stack driver also.

job ID is

2018-03-09_00_40_49-15224076611250277770

2018-02-12_22_35_23-4736481063361562693

What's the problem?

And, How can I know when this drainig job complete? (predict time)

Upvotes: 2

Views: 2493

Answers (2)

p13rr0m
p13rr0m

Reputation: 1297

Streaming pipelines that throw unhandled exceptions while processing data currently can not be drained.

https://issuetracker.google.com/issues/147124216

https://cloud.google.com/dataflow/docs/resources/faq#how-are-java-exceptions-handled-in-cloud-dataflow

Upvotes: 1

dstanco
dstanco

Reputation: 61

Have you tried to update your job with higher machine type? Maybe it is just a too heavy process for it to drain (lots of data is included).

Upvotes: 0

Related Questions