CODEWITHSUNDEEP
pythonlinuxjupyter-notebookgoogle-colaboratory
Chan Kha Vu
Chan Kha Vu

Reputation: 10400

How can I prevent Google Colab from disconnecting?

Is there a way to programmatically prevent Google Colab from disconnecting on a timeout?

The following describes the conditions causing a notebook to automatically disconnect:

Google Colab notebooks have an idle timeout of 90 minutes and absolute timeout of 12 hours. This means, if user does not interact with his Google Colab notebook for more than 90 minutes, its instance is automatically terminated. Also, maximum lifetime of a Colab instance is 12 hours.

Naturally, we want to automatically squeeze the maximum out of the instance, without having to manually interact with it constantly. Here I will assume commonly seen system requirements:

I should point out here that such behavior does not violate Google Colab's Terms of Use, although it is not encouraged according to their FAQ (in short: morally it is not okay to use up all of the GPUs if you don't really need it).

My current solution is very dumb:

Are there better ways?

Upvotes: 353

Views: 470751

Answers (30)

Alvin Kimata
Alvin Kimata

Reputation: 338

As of June 2023, here's the working javascript code for preventing Google colab from disconnecting due to inactivity.

 function keepAliveProgrammatically() {
  document.querySelector('colab-connect-button').shadowRoot.querySelector("#connect").click();
}

The function above is responsible for clicking the compute resources button It is clicked after every 6 seconds. The code snippet below is responsible for running the function.

const keepAliveProgrammaticallyInterval = setInterval(() => {
    keepAliveProgrammatically();
}, 6000);

To stop the code, use the code snippet below.

clearInterval(keepAliveProgrammaticallyInterval);

Upvotes: 10

mikey
mikey

Reputation: 2674

This snippet when run in colab or kaggle seems to work:

from IPython.display import Audio
import numpy as np

Audio(np.array([0] * 2 * 3600 * 3000, dtype=np.int8), normalize=False, rate=3000, autoplay=True)

# Audio([None] * 2 * 3600 * 3000, normalize=False, rate=3000, autoplay=True)

Explanation: the basic idea is to generate a large engouh numpy array and play it as an audioclip with a counter running.

  • [None] * 2 * 3600 * 3000: enough values to play for 2 hours to keep the session alive
  • dtype=np.int8: allocate a single byte, it's all zero anyway.
  • 2 * 3600: 2 hours, 3000 is the smallest sample rate (see below) to enable a running counter.
  • rate: sample rate
  • normalize=False: since it's all zero, the default normalize=True will result in a divide by zerro warning
  • autoplay=True: no need to click to play the clip.

It takes a few seconds for the widget to appear and another few seconds for the clip to start autoplaying. enter image description here

Data for one hour (np.array([0] * 2 * 3600 * 3000, dtype=np.int8)) appears to require 100M RAM.

The alternative (Audio([None] * 2 * 3600 * 3000, normalize=False, rate=3000, autoplay=True)) wont need the numpy lib but will require more RAM.

Audio can also play a local audiofile (in colab or kaggle) or something from a url. Check Audio's doc for more details.

Upvotes: 0

A. Mair
A. Mair

Reputation: 61

Version July 2023:

function ConnectButton() {
    console.log("trying to click connect button");

    let colabConnectButton = document.querySelector("colab-connect-button");
    if (colabConnectButton && colabConnectButton.shadowRoot) {
        let actualButton = colabConnectButton.shadowRoot.querySelector("colab-toolbar-button#connect");
        if (actualButton) {
            actualButton.click();
            console.log("connect button clicked");
        } else {
            console.log("button not found");
        }
    } else {
        console.log("button not found");
    }
}
setInterval(ConnectButton, 60000);

Upvotes: 4

kiranr
kiranr

Reputation: 2465

You can bookmark the notebook to make it stay connected:

function ClickConnect(){
    console.log("Clicked on star button"); 
    document.querySelector("iron-icon#star-icon").click()
}
setInterval(ClickConnect, 60000)

Now you can see the blinking of the star every minute.

Upvotes: 8

ali khabazian
ali khabazian

Reputation: 5

This is just for people who work with the Ipywidget library. If you don't, it may not be useful for you.

Prevent using Ipywidgets.

Ipywidgets and similar packages that they prepare buttons, text fields, etc., complete a cell after a run in Jupiter or Colab.

Colab saves the last time you ran a cell and if you train your model with these packages then it thinks that you are idle, so after 30 minutes it will make you disconnect.

Like this

Upvotes: -2

Joaquim
Joaquim

Reputation: 440

The solution in the YouTube video "How to prevent Google Colab from disconnecting | A simple solution" worked for me.

Install the pynput library that allows you to control and monitor input devices.

pip install pynput

Now execute this code on your local machine and place the mouse cursor in an empty cell in the Colab notebook being run.

from pynput.mouse import Controller,Button
import time

mouse = Controller()

while True:
    mouse.click(Button.left,1)
    print('clicked')

    time.sleep(5)

Upvotes: 7

Luciano Dourado
Luciano Dourado

Reputation: 501

Updating (July 2021).

function ConnectButton(){ 
    console.log("Working"); 
    document.querySelector("#connect").click() 
}
setInterval(ConnectButton, 60000);

Upvotes: 9

Diptesh Chakraborty
Diptesh Chakraborty

Reputation: 420

Instead of clicking the Connect button, I just clicking on the Comment button to keep my session alive (August 2020):

function ClickConnect(){

    console.log("Working");
    document.querySelector("#comments > span").click()
}
setInterval(ClickConnect,5000)

Upvotes: 25

Harsh Goyal
Harsh Goyal

Reputation: 83

This works for me:

function ClickConnect(){
    console.log("Working"); 
    document.querySelector("paper-icon-button").click()
}
Const myjob = setInterval(ClickConnect, 60000)

If isn't working, try clear it by running:

clearInterval(myjob)

Upvotes: 6

Rajesh Kumbhakar
Rajesh Kumbhakar

Reputation: 59

Well, I am not a Python guy nor I do know what the actual use of this 'Colab' is. I use it as a build system, lol. And I used to setup ssh forwarding in it and then put this code and just leave it running and yeah it works.

import getpass
authtoken = getpass.getpass()

Upvotes: 3

Li-Pin Juan
Li-Pin Juan

Reputation: 1245

The JavaScript code below works for me. Credits to @artur.k.space.

function ColabReconnect() {
    var dialog = document.querySelector("colab-dialog.yes-no-dialog");
    var dialogTitle = dialog && dialog.querySelector("div.content-area>h2");
    if (dialogTitle && dialogTitle.innerText == "Runtime disconnected") {
        dialog.querySelector("paper-button#ok").click();
        console.log("Reconnecting...");
    } else {
        console.log("ColabReconnect is in service.");
    }
}
timerId = setInterval(ColabReconnect, 60000);

In the Colab notebook, press Ctrl + Shift + the I key simultaneously. Copy and paste the script into the prompt line. Then hit Enter before closing the editor.

By doing so, the function will check every 60 seconds to see if the onscreen connection dialog is shown, and if it is, the function would then click the OK button automatically for you.

Upvotes: 4

hashc
hashc

Reputation: 39

The following latest solution works for me:

function ClickConnect(){
  colab.config
  console.log("Connnect Clicked - Start"); 
  document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click();
  console.log("Connnect Clicked - End");
};
setInterval(ClickConnect, 60000)

Upvotes: 3

code-freeze
code-freeze

Reputation: 485

function ClickConnect()
{
    console.log("Working...."); 
    document.querySelector("paper-button#comments").click()
}
setInterval(ClickConnect, 600)

This worked for me, but use it wisely.

Upvotes: 2

singrium
singrium

Reputation: 3026

I was looking for a solution until I found a Python 3 script that randomly moved the mouse back and forth and clicked, always on the same place. But that's enough to fool Colaboratory into thinking I'm active on the notebook, and it will not disconnect.

import numpy as np
import time
import mouse
import threading

def move_mouse():
    while True:
        random_row = np.random.random_sample()*100
        random_col = np.random.random_sample()*10
        random_time = np.random.random_sample()*np.random.random_sample() * 100
        mouse.wheel(1000)
        mouse.wheel(-1000)
        mouse.move(random_row, random_col, absolute=False, duration=0.2)
        mouse.move(-random_row, -random_col, absolute=False, duration = 0.2)
        mouse.LEFT
        time.sleep(random_time)


x = threading.Thread(target=move_mouse)
x.start()

You need to install the needed packages: sudo -H pip3 install <package_name>

You just need to run it (in your local machine) with sudo (as it takes control of the mouse) and it should work, allowing you to take full advantage of Colaboratory's 12h sessions.

Credits: For those using Colab (Pro): Preventing Session from disconnecting due to inactivity

Upvotes: 2

Albos Hajdari
Albos Hajdari

Reputation: 121

I have a problem with these JavaScript functions:

function ClickConnect(){
    console.log("Clicked on connect button"); 
    document.querySelector("colab-connect-button").click()
}
setInterval(ClickConnect, 60000)

They print the "Clicked on connect button" on the console before the button is actually clicked.

As you can see from different answers to this question, the id of the connect button has changed a couple of times since Google Colab was launched. And it could be changed in the future as well.

So if you're going to copy an old answer to this question, it may say "Clicked on connect button", but it may actually not do that. Of course, if the clicking won't work, it will print an error on the console, but what if you may not accidentally see it?

So you better do this:

function ClickConnect(){
    document.querySelector("colab-connect-button").click()
    console.log("Clicked on connect button"); 
}
setInterval(ClickConnect,60000)

And you'll definitely see if it truly works or not.

Upvotes: 3

divyansh srivastava
divyansh srivastava

Reputation: 49

function ClickConnect(){
    console.log("Clicked on connect button"); 
    document.querySelector("connect").click() // Change id here
}
setInterval(ClickConnect,60000)

Try the above code. It worked for me:)

Upvotes: 1

Jinhua Wang
Jinhua Wang

Reputation: 1759

I would recommend using jQuery (it seems that Colaboratory includes jQuery by default).

function ClickConnect(){
  console.log("Working");
  $("colab-toolbar-button").click();
}
setInterval(ClickConnect, 60000);

Upvotes: 3

Stephane Belemkoabga
Stephane Belemkoabga

Reputation: 51

This one worked for me (it seems like they changed the button classname or id):

function ClickConnect(){
    console.log("Working");
    document.querySelector("colab-connect-button").click()
}
setInterval(ClickConnect,60000)

Upvotes: 5

esr
esr

Reputation: 75

I tried the code in previous answers, but they did not work for me. So here is my JavaScript code for reconnecting.

let interval = setInterval(function(){
let ok = document.getElementById('ok');
if(ok != null){
   console.log("Connect pushed");
ok.click();
}},60000)

You can use it with the same way (run it on the console of your browser) to run it.

If you want to stop the script, you can enter clearInterval(interval) and want to run again setInterval(interval).

Upvotes: 6

Hu Xixi
Hu Xixi

Reputation: 2157

The previous answers with the help of some scripts maybe work well.

I have a solution (or a kind of trick) for that annoying disconnection without scripts, especially when your program must read data from your google drive, like training a deep learning network model, where using scripts to do the reconnect operation is of no use, because once you disconnect with your Colaboratory, the program is just dead. You should manually connect to your Google Drive again to make your model able to read dataset again, but the scripts will not do that thing.

I've already tested it many times and it worked well.

When you run a program on the Colaboratory page with a browser (I use Chrome), just remember that don't do any operation to your browser once your program starts running, like: switch to other webpages, open or close another webpage, and so on. Just just leave it alone there and waiting for your program finish running. You can switch to another software, like PyCharm to keep writing your code, but not switch to another webpage.

I don't know why open or closing or switching to other pages will cause the connection problem of the Colaboratory page, but each time I try to bothered my browser, like do some search job, my connection to Colaboratory will soon break down.

Upvotes: 11

Seyon Seyon
Seyon Seyon

Reputation: 567

Create Python code on your PC with pynput:

from pynput.mouse import Button, Controller
import time

mouse = Controller()

while True:
    mouse.click(Button.left, 1)
    time.sleep(30)

Run this code on your desktop. Then point the mouse arrow over (Colaboratory's left panel - file section) directory structure on any directory.

This code will keep clicking on the directory every 30 seconds, so it will expand and shrink every 30 seconds. So your session will not get expired.

Important - you have to run this code on your PC.

Upvotes: 29

Kavyajeet Bora
Kavyajeet Bora

Reputation: 652

Well, this is working for me:

Run the following code in the console and it will prevent you from disconnecting.

Ctrl + Shift + I to open the inspector view. Then go to the console.

function ClickConnect(){
    console.log("Working");
    document.querySelector("colab-toolbar-button#connect").click()
}
setInterval(ClickConnect,60000)

How to prevent Colaboratory from disconnecting

Upvotes: 30

M. Fuller
M. Fuller

Reputation: 81

I don't believe the JavaScript solutions work anymore. I was doing it from within my notebook with:

    from IPython.display import display, HTML
    js = ('<script>function ConnectButton(){ '
           'console.log("Connect pushed"); '
           'document.querySelector("#connect").click()} '
           'setInterval(ConnectButton,3000);</script>')
    display(HTML(js))

When you first do a Run all (before the JavaScript or Python code has started), the console displays:

Connected to
wss://colab.research.google.com/api/kernels/0e1ce105-0127-4758-90e48cf801ce01a3/channels?session_id=5d8...

However, ever time the JavaScript runs, you see the console.log portion, but the click portion simply gives:

Connect pushed

Uncaught TypeError: Cannot read property 'click' of null
 at ConnectButton (<anonymous>:1:92)

Others suggested the button name has changed to #colab-connect-button, but that gives same error.

After the runtime is started, the button is changed to show RAM/DISK, and a drop down is presented. Clicking on the drop down creates a new <DIV class=goog menu...> that was not shown in the DOM previously, with 2 options "Connect to hosted runtime" and "Connect to local runtime". If the console window is open and showing elements, you can see this DIV appear when you click the dropdown element. Simply moving the mouse focus between the two options in the new window that appears adds additional elements to the DOM, as soon as the mouse looses focus, they are removed from the DOM completely, even without clicking.

Upvotes: 8

SciPy
SciPy

Reputation: 6100

Using Python with Selenium:

from selenium.webdriver.common.keys import Keys
from selenium import webdriver
import time

driver = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver')

notebook_url = ''
driver.get(notebook_url)

# run all cells
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + Keys.F9)
time.sleep(5)

# click to stay connected
start_time = time.time()
current_time = time.time()
max_time = 11*59*60 #12hours

while (current_time - start_time) < max_time:
    webdriver.ActionChains(driver).send_keys(Keys.ESCAPE).perform()
    driver.find_element_by_xpath('//*[@id="top-toolbar"]/colab-connect-button').click()
    time.sleep(30)
    current_time = time.time()

Upvotes: 7

Tanay Karve
Tanay Karve

Reputation: 2285

As of March 2021, none of these methods will work as Google added a CAPTCHA button that randomly pops up after some time.

Prior to that, the solution was very easy, and didn't need any JavaScript. Just create a new cell at the bottom having the following line:

while True:pass

Now keep the cell in the run sequence so that the infinite loop won't stop and thus keep your session alive.

Old method:

Set a JavaScript interval to click on the connect button every 60 seconds.

Open developer-settings (in your web-browser) with Ctrl+Shift+I then click on console tab and type this on the console prompt. (for mac press Option+Command+I)

function ConnectButton(){
  console.log("Connect pushed");
  document.querySelector("#top-toolbar > colab-connectbutton").shadowRoot.querySelector("#connect").click()
}
setInterval(ConnectButton,60000);

Upvotes: 211

MrMartin
MrMartin

Reputation: 467

When all else fails, you can get background execution by subscribing to Google Colab Pro+

Upvotes: -3

Some53
Some53

Reputation: 73

Okay I've found a nifty solution that will get rid of the

Active session

pop up automatically. We'll need 2 function for that,

same procedure as earlier. Inspect> console > paste function one by one

1

function clickConnect() {
    try {
         document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click();
        // this also works, if above one doesn't work, comment it and uncomment below one
        //document.querySelector("colab-connect-button").shadowRoot.getElementById('connect').click();
        setTimeout(clickDismiss, 500);
        console.log("Keeping Colab Alive!");    
    } catch (error) {
        console.log(error);
    }
}

2

function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

3

async function clickDismiss() {
    
    try {   
        
        // click manage session button
        document.querySelector("colab-usage-display").shadowRoot.querySelector("paper-button").click();
    
        } catch (error) {
        console.log(error);
    }
    
        try {
            // leave from manage session window
            await sleep(1000);
            document.querySelector('colab-sessions-dialog').shadowRoot.querySelector('.dismiss').click();
                } catch (error) {
        console.log(error);
    }
    
        try {   
            // click close button
            await sleep(1000);
            document.querySelector("paper-tab").querySelector("paper-icon-button").shadowRoot.getElementById('icon').click();
                } catch (error) {
        console.log(error);
    }
    
}

4

setInterval(ClickConnect, 60000);

EDIT:

So if you don't like doing all this stuff manually there is a way to automate all this!

Way_1. Use this Chrome Extension and done

or

Way_2.

  1. Use Page-Manipulator extension
  2. Click on it then click add Java Script button > + New + > filename
  3. Say file name is ColabAlive > Make
  4. Active Website = colab.research.google.com
  5. Matching pages = Recursive
  6. Then use below code,

// 1
function clickConnect() {
    try {
        document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click();
        // this also works, if above one doesn't work, comment it and uncomment below one
        //document.querySelector("colab-connect-button").shadowRoot.getElementById('connect').click();
        setTimeout(clickDismiss, 500);
        console.log("Keeping Colab Alive!");
    } catch (error) {
        console.log(error);
    }
}

//2
function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

//3
async function clickDismiss() {

    try {

        // click manage session button
        document.querySelector("colab-usage-display").shadowRoot.querySelector("paper-button").click();

    } catch (error) {
        console.log(error);
    }

    try {
        // leave from manage session window
        await sleep(1000);
        document.querySelector('colab-sessions-dialog').shadowRoot.querySelector('.dismiss').click();
    } catch (error) {
        console.log(error);
    }

    try {
        // click close button
        await sleep(1000);
        document.querySelector("paper-tab").querySelector("paper-icon-button").shadowRoot.getElementById('icon').click();
    } catch (error) {
        console.log(error);
    }

}
//4 
setInterval(clickConnect, 60000);

  1. Click Active and then reload > done

Credit goes to Oshayr, Albert Einstein and everyone who posted their solution here.

Upvotes: 4

Nitesh Jindal
Nitesh Jindal

Reputation: 780

Since the id of the connect button is now changed to "colab-connect-button", the following code can be used to keep clicking on the button.

function ClickConnect(){
    console.log("Clicked on connect button"); 
    document.querySelector("colab-connect-button").click()
}
setInterval(ClickConnect,60000)

If still, this doesn't work, then follow the steps below:

  1. Right-click on the connect button (on the top-right side of the colab)
  2. Click on inspect
  3. Get the HTML id of the button and substitute in the following code
function ClickConnect(){
    console.log("Clicked on connect button"); 
    document.querySelector("Put ID here").click() // Change id here
}
setInterval(ClickConnect,60000)

Upvotes: 68

GarouDan
GarouDan

Reputation: 3833

var startColabHandler = function startColabHandler(interval = 60000, enableConnectButton = false) {
    console.log("colabHandler - configure - start: " + new Date());
    
    var colabClick = function colabClick() {
        console.log("colabHandler - click - start: " + new Date());
        
        if (enableConnectButton === true) {
            var button1 = document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect");
            
            if (button1) {
                button1.click()
            }
        }
        
        button2 = document.querySelector("html body colab-dialog.yes-no-dialog paper-dialog div.buttons paper-button#ok");
        
        if (button2) {
            button2.click()
        }
        
        console.log("colabHandler - click - end: " + new Date());
    };

    var intervalId = setInterval(colabClick, interval);

    window.stopColabHandler = function stopColabHandler() {
        console.log("colabHandler - stop - start: " + new Date());
        
        clearInterval(intervalId);
        
        console.log("colabHandler - stop - start: " + new Date());
    };

    console.log("colabHandler - configure - end: " + new Date());
};

Upvotes: 2

MutexMonk
MutexMonk

Reputation: 123

Try this to avoid all the annoying dialog boxes appearing while you work when trying to simulate the click on the toolbar connect button every minute. you can just copy paste this to your console, call the method and you can work on your notebook.

function connectRefresher() {
       window.ConnectButtonIntervalId = setInterval(function ConnectButton(){
                console.log("connected"); 
                document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click();
                document.querySelector("colab-sessions-dialog").shadowRoot.querySelector("#footer > div > paper-button").click();
                console.log("closed the dialog!!"); 
            },60000);
    }
    
function clearRefresher() { 
           console.log("clear Interval called !!");
           clearInterval(window.ConnectButtonIntervalId);
    }

 connectRefresher(); //to connect the refresher
 clearRefresher(); //to disconnect the refresher

Upvotes: 3

Related Questions