CODEWITHSUNDEEP
google-cloud-platformjupyter-notebookgoogle-compute-enginegoogle-cloud-datalabgoogle-earth-engine
Nicolò Franceschetti
Nicolò Franceschetti

Reputation: 11

Datalab notebook don't utilize all Compute Engine resources

I try run a notebook in Google Cloud Platform using Compute Engine virtual machine. I tested operations contained in the notebook,that refer to Earth Engine functions and objects, in Earth Engine Code Editor and everything goes well in decent computational time. Now I export all the operation in datalab notebook, push my container on Cloud Platform and run it on a virtual machine of standard type: n1-standard-8. The operation doesn't go faster respect to when I run the datalab container on my local machine and If I visualize the CPU's usage on the virtual machine it's about 5%, notebook is stopped for exceeding computation time. I don't understand why the VM doesn't utilize more CPU's computation resources and if I can use a command for extend notebook computation time.

Upvotes: 1

Views: 321

Answers (1)

Milad Tabrizi
Milad Tabrizi

Reputation: 498

By running stress command to inflict configurable of CPU stress on the system, you can make sure that the issue is related to your machine on GCP or your resources.

After running the stress command in your linux machine, if your CPU usage reach over 90%, means your VM is working fine and you need to check your application resources to find why restricted and can not use CPU more than 5%.

As you are using n1-standard-8 type machine, means you have 8 virtual CPUs and 30 GB of memory.

Example to stress 8 cores for 90 seconds:

 stress --cpu 8 --timeout 90

You can go to your VM instances page and select your VM and then check your machine’s graph to see how many percent of your CPU has been used.

If you observe that CPU usage did not change after running stress on the graph, make sure that you did not run nice, cpulimit, and cgroups tools in your machine to restrict the CPU.

Upvotes: 1

Related Questions