CODEWITHSUNDEEP
google-cloud-dataflow
Lukasz Cwik
Lukasz Cwik

Reputation: 1731

What happens when I change pipeline options while executing on the Dataflow worker?

What happens when if I change pipeline options while executing on the Dataflow worker?

The public documentation for PipelineOptions is: https://cloud.google.com/dataflow/java-sdk/JavaDoc/com/google/cloud/dataflow/sdk/options/PipelineOptions

Upvotes: 0

Views: 269

Answers (1)

Lukasz Cwik
Lukasz Cwik

Reputation: 1731

PipelineOptions is safe to access from multiple threads and is global per Dataflow worker. You should only get values from PipelineOptions and not update PipelineOptions or mutate any values stored within PipelineOptions on the Dataflow Worker. This means that any object placed within PipelineOptions must be thread-safe as it may be accessed from many threads at the same time. Note that there are many types within Java such as SimpleDateFormat which may seem to be thread-safe but are not.

Upvotes: 2

Related Questions