CODEWITHSUNDEEP
google-compute-enginegoogle-cloud-dataproc
Daniel Osielczak
Daniel Osielczak

Reputation: 11

Shrink a Dataproc worker boot disk

Due to some mix-up during planning we ended up with several worker nodes running 23TB drives which are now almost completely unused (we keep data on external storage). As the drives are only wasting money at the moment, we need to shrink them to a reasonable size. Using weresync I was able to fully clone the drive to a much smaller one but apparently you can't swap the boot drive in GCE (which makes no sense to me). Is there a way to achieve that or do I need to create new workers using the images? If so, is there any other config I need to copy to the new instance in order for it to be automatically joined to the cluster?

Upvotes: 1

Views: 309

Answers (2)

Daniel Osielczak
Daniel Osielczak

Reputation: 11

I ended up creating a ticket with GCP support - https://issuetracker.google.com/issues/120865687 - to get an official answer to that question. Got an that this is not possible currently but should be available shortly (within months) in the beta GCP CLI, possibly in the Console on a later data as well. Went on with a complete rebuild of the cluster.

Upvotes: -1

Igor Dvorzhak
Igor Dvorzhak

Reputation: 4455

Dataproc does not support VMs configuration changes in running clusters.

I would advise you to delete old cluster and create new one with workers disk size that you need.

Upvotes: 1

Related Questions