CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-dataproc
gogasca
gogasca

Reputation: 10058

Dataproc change worker size

My jobs (ML jobs) require more than 15GB RAM per worker. How to change the machine type for worker? Currently:

n1-standard-4 (4 vCPU, 15.0 GB memory)

I would prefer to keep my cluster not recreate it.

Upvotes: 0

Views: 787

Answers (2)

rsantiago
rsantiago

Reputation: 2099

I'm afraid it can be done following these steps:

  1. Go to the VM instances section in the Compute Engine -> Instances
  2. Stop the instance
  3. Change the type machine
  4. Start it

I have just done it and it worked. If a worker is shut down the cluster will continue working, but if you modify the master the cluster may not work after the change, in any case I recommend restart your cluster. Just be careful on doing this because data stored may be removed according to Dataproc documentation.

Upvotes: 2

cyxxy
cyxxy

Reputation: 608

You cannot change the machine type for an existing cluster - this is not something Dataproc supports. What is the reason you want to keep your cluster rather than create a new one?

Upvotes: 2

Related Questions