CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-dataproc
thisdudeiknew
thisdudeiknew

Reputation: 75

Google Cloud Dataproc: cluster create errors (debconf DbDriver config.dat locked)

Recently, I have experienced occasional errors while attempting to create dataproc clusters in GCP. The creation command is similar to:

gcloud dataproc clusters create ${CLUSTER_NAME} \
       --zone "us-east1-b" \
       --master-machine-type "n1-standard-16" \
       --master-boot-disk-size 150 \
       --num-workers ${WORKER_NODE_COUNT:-9} \
       --worker-machine-type "n1-standard-16" \
       --worker-boot-disk-size 25 \
       --project ${PROJECT_NAME}  \
       --properties 'yarn:yarn.log-aggregation-enable=true'

Very intermittently, the error I receive is:

ERROR: (gcloud.dataproc.clusters.create) Operation [projects/PROJECT/regions/global/operations/UUID] failed: Multiple Errors:
- Failed to initialize node random-name-m. See output in: gs://dataproc-UUID-us/google-cloud-dataproc-metainfo/UUID/random-name-m/dataproc-startup-script_output
- Failed to initialize node random-name-w-0. See output in: gs://dataproc-UUID-us/google-cloud-dataproc-metainfo/UUID/random-name-w-0/dataproc-startup-script_output
- Failed to initialize node random-name-w-1. See output in: gs://dataproc-UUID-us/google-cloud-dataproc-metainfo/UUID/random-name-w-1/dataproc-startup-script_output
- Worker random-name-w-8 unable to register with master random-name-m. This could be because it is offline, or network is misconfigured..

And the last lines of the Google Storage bucket output file (dataproc-startup-script_output) are:

+ debconf-set-selections
debconf: DbDriver "config": /var/cache/debconf/config.dat is locked by another process: Resource temporarily unavailable
++ logstacktrace
++ local err=1
++ local code=1
++ set +o xtrace
ERROR: 'debconf-set-selections' exited with status 1
Call tree:
 0: /usr/local/share/google/dataproc/startup-script-cloud_datarefinery_image_20180803_nightly-RC04.sh:490 main
Exiting with status 1

This one is really starting to annoy me! Any ideas/thoughts/resolutions are much appreciated!

Upvotes: 2

Views: 370

Answers (1)

Mikayla Konst
Mikayla Konst

Reputation: 111

A fix for this issue will be rolling out over the course of next week's release. You can check the release notes to see when the fix has rolled out here: https://cloud.google.com/dataproc/docs/release-notes

Upvotes: 1

Related Questions