CODEWITHSUNDEEP
apache-beamdataflow
Kenta Kozuka
Kenta Kozuka

Reputation: 47

Dataflow: set DataflowPipelineDebugOptions

My pipeline gives OOM errors constantly so I read a fowllowing answer and try to set --dumpHeapOnOOM and --saveHeapDumpsToGcsPath. But it seems that these options do not work. Do I need to change my code or modify something else?

Memory profiling on Google Cloud Dataflow

Upvotes: 1

Views: 829

Answers (1)

Rui Wang
Rui Wang

Reputation: 839

You will want to check configuring-pipeline-options.

The current way in Apache Beam (2.9.0) to configure pipeline option in command line is --<option>=<value>.

In your case, you can set --dumpHeapOnOOM=true --saveHeapDumpsToGcsPath="gs://foo"

Upvotes: 3

Related Questions