CODEWITHSUNDEEP
google-cloud-dataflow
rav
rav

Reputation: 3688

Dataflow job-id as a "lock" mechanism

Is it (would it be) possible to use dataflow job-id as a "lock" mechanism (be able to specify/lock job reference) - kinda the same way one can insert BQ job reference?

Upvotes: 1

Views: 209

Answers (1)

Ben Chambers
Ben Chambers

Reputation: 6130

The job ID is a unique identifier for a specific invocation of a pipeline. Since it is unique, using it as a lock won't work. suitable for use as a lock.

If you're trying to have one instance of a given pipeline running at a time, you can use the job name. Only one instance of a pipeline with a specific name can be running at a time (see getJobName).

You can set the name with --jobName=<some job name>.

Upvotes: 1

Related Questions