CODEWITHSUNDEEP
javagoogle-cloud-dataflow
Valentin
Valentin

Reputation: 365

How do I drain a pipeline from within another pipeline?

I need to programmatically drain a pipeline from within another pipeline. The DataflowPipelineJob class doesn't have a drain method implemented. Is it possible to call drain on a pipeline from with the Java API?

Upvotes: 6

Views: 1097

Answers (1)

Valentin
Valentin

Reputation: 365

Was able to initiate draining with the following code:

    // spawn child pipe
    DataflowPipelineRunner runner = DataflowPipelineRunner.fromOptions(options);
    DataflowPipelineJob job = runner.run(p);

    // under the some condition later, drain the spawned pipe:
    Dataflow client = com.google.cloud.dataflow.sdk.util.Transport.newDataflowClient(options).build();
    Job content = new Job();
    content.setProjectId(options.getProject());
    content.setId(job.getJobId());
    content.setRequestedState("JOB_STATE_DRAINING");
    client.projects().jobs()
            .update(options.getProject(), job.getJobId(), content)
            .execute();

Upvotes: 6

Related Questions