CODEWITHSUNDEEP
google-colaboratory
Dylan Kerler
Dylan Kerler

Reputation: 2187

What is the official way to do long-running training on google colab?

I have a neural net that takes about 7-15 days to train on several GPU's. Google colab disconnects after running the script for a few hours. There are a couple "hacks" that you can do to keep the session alive but this is obviously not the official standard.

Once I have written my script in google colab, how should I go about running the script for a long period of time? I'm connected and using google's deep learning VM's.

Upvotes: 9

Views: 11205

Answers (1)

desertnaut
desertnaut

Reputation: 60318

There is currently no way of running scripts for such long times (i.e. days) in the free version of Colab; in fact, it is clear from the Resource Limits section of the official FAQ that the maximum running time is 12 hours (emphasis added):

How long can notebooks run in Colab?

Notebooks run by connecting to virtual machines that have maximum lifetimes that can be as much as 12 hours. Notebooks will also disconnect from VMs when left idle for too long. Maximum VM lifetime and idle timeout behavior may vary over time, or based on your usage. This is necessary for Colab to be able to offer computational resources for free. Users interested in longer VM lifetimes and more lenient idle timeout behaviors that don’t vary as much over time may be interested in Colab Pro.

So, if you really need running times in the order of days, you should consider Colab Pro.

Upvotes: 5

Related Questions