CODEWITHSUNDEEP
google-cloud-platformscheduled-tasksgoogle-colaboratory
Pranay
Pranay

Reputation: 91

Automated execution of a Colab notebook on a recurring schedule

Is there a reliable way of executing a Google Colab notebook at a specific schedule ? This recurring schedule should not require me to open my laptop to execute or trigger it in any way.

I also have a free trial of GCP as well in case that is helpful.

Thank you for taking the time to read this !

Upvotes: 7

Views: 10545

Answers (2)

NsaNinja
NsaNinja

Reputation: 314

You can use Selenium for this one. It may be overkill but you can use it for free.

import time
import os
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains

COLAB_URL = 'https://colab.research.google.com/drive/CHANGE'
DOWNLOAD_FOLDER = 'CHANGEME/Downloads/' 
OUTPUT_FILE_NAME = 'copy.txt'

def run_colab(COLAB_URL=COLAB_URL, DOWNLOAD_FOLDER=DOWNLOAD_FOLDER, OUTPUT_FILE_NAME=OUTPUT_FILE_NAME):
    chrome_options = Options()
    chrome_options.add_argument("--headless")
    chrome_options.add_argument("--no-sandbox")
    chrome_options.add_argument("--disable-dev-shm-usage")
    chrome_options.add_argument(f"--download.default_directory={DOWNLOAD_FOLDER}")
    chrome_options.add_argument("--user-data-dir=CHANGEME/.config/google-chrome/Default") # Added line

    wd = webdriver.Chrome(options=chrome_options)
    wd.get(COLAB_URL)
    print(wd.title)
    print("Page loaded")

    wait = WebDriverWait(wd, 10)
    runtime_button = wait.until(EC.presence_of_element_located((By.ID, 'runtime-menu-button')))
    runtime_button.click()
    action_chains = ActionChains(wd)
    action_chains.key_down(Keys.CONTROL).send_keys(Keys.F9).key_up(Keys.CONTROL).perform()
    print("Running colab...")
    print("Waiting for the task to complete...")

    while not os.path.isfile(os.path.join(DOWNLOAD_FOLDER, OUTPUT_FILE_NAME)):
        time.sleep(1)

    print("Task completed")

    time.sleep(5)

    print("Closing Driver...")

    wd.quit()

if __name__ == "__main__":
    run_colab()

Also Jupyter notebook code

from google.colab import files
import time
with open("copy.txt", "w") as file:
    file.write("Hello")

files.download("copy.txt")

After that you can create cronjob for python script.

0 8 * * * /usr/bin/python3 /path_to_your_script/run_colab.py

This example would run the script every day at 8:00 AM. Adjust the timing as needed.

Upvotes: 1

rchurt
rchurt

Reputation: 1533

This is a feature that the team at Google Colab is considering adding, but you can already schedule notebooks to run on GCP: https://cloud.google.com/blog/products/application-development/how-to-schedule-a-recurring-python-script-on-gcp

Upvotes: 7

Related Questions