CODEWITHSUNDEEP
google-cloud-dataflowapache-beam
varun r
varun r

Reputation: 204

Enabling Debug|Trace worker logs for google cloud dataflow

Not able to enable the debug|trace level logging of the dataflow workers.

The documentation :https://cloud.google.com/dataflow/docs/guides/logging#SettingLevels

indicates the usage of DataflowWorkerLoggingOptions to programmatically overrides the default log level on the worker and enable the debug|trace level logging; however the interface is deprecated and no more present in bean-sdk 2.27.0 .

Has anyone been able to enable the worker level debugging in cloud dataflow; in any way.

Upvotes: 0

Views: 1074

Answers (1)

Kenn Knowles
Kenn Knowles

Reputation: 6023

The documentation is still up to date and the interface is still present and will work.

The interface is deprecated because the Java-based Dataflow worker is not used when running a pipeline using Beam's portability framework. Quoting the deprecation message:

@deprecated This interface will no longer be the source of truth for worker logging configuration once jobs are executed using a dedicated SDK harness instead of user code being co-located alongside Dataflow worker code. Please set the option below and also the corresponding option within org.apache.beam.sdk.options.SdkHarnessOptions to ensure forward compatibility.

So what you should do is follow the instructions that you linked and also set up logging in SdkHarnessOptions.

Upvotes: 0

Related Questions