CODEWITHSUNDEEP
pythonjupyter-notebookairflowgoogle-colaboratory
Sado
Sado

Reputation: 447

Running google colab every day at a specific time

I recently have built a Python program that runs on Google Colaboratory, I need to run the program every day at a specific time, So Is there any way to schedule it to run on Google Colab?

Upvotes: 42

Views: 68311

Answers (5)

nullstellensatz
nullstellensatz

Reputation: 774

Scheduling scripts seems to now be natively supported in Google Colab (if you are willing to pay): https://github.com/googlecolab/colabtools/wiki/Scheduled-notebooks

Upvotes: 3

Lucas GCode
Lucas GCode

Reputation: 39

You can use DeepNote, even the free plan allows to schedule your scripts.

Upvotes: 2

Prakhar Agarwal
Prakhar Agarwal

Reputation: 2842

You need to create a notebooks.csv listing all the Colaboratory URLs. Then use colabctl to run each notebook (In order, synchronously mentioned in the CSV) and then pauses for a period of n seconds of time before running them again.

You can then run python colabctl.py <end-string> <sleep-seconds>,

There's a gCookies.pkl file in the repo. Ensure that your local Python version is the same one as used to create this serialized pkl file.

Ref: https://github.com/TensorTom/colabctl

Now that we have a notebook and script ready, next is to be able to SSH into Google colab.

  • create an account on ngrok.com
  • Copy and paste below code in colab which installs ngrok and creates a tunnel for us

Source : https://gist.github.com/yashkumaratri/204755a85977586cebbb58dc971496da#file-google-colab-ssh

  • Now, on your local machine, open terminal and SSH to connect to the Colab System.

ssh [email protected] -p <port_num_generated_above>

Now you have access to the whole system through SSH terminal. And can continue to use colabctl to any other corn script on local environment to execute your remote colab script.

Upvotes: 23

Donald S
Donald S

Reputation: 1753

If you are willing to pay for this, you can use the Google Cloud scheduler:

https://cloud.google.com/scheduler

Related links for setup details:

https://minimaxir.com/2018/11/cheap-cron/

https://cloud.google.com/scheduler/docs/quickstart

If not, in addition to the previous answer, you can consider moving the script to your local environment and using Task Scheduler.

Upvotes: 10

Sy Ker
Sy Ker

Reputation: 2190

You can use colabctl to run your Google Colaboratory notebooks from the command line, on a schedule if you wish. Simply add colabctl.py to a cron job.

Note: If you want to run it on a headless server, you will first need to run it once from your desktop to save your login cookie to the pickle file and then upload that pickle to your server along with colabctl.

Upvotes: 3

Related Questions