CODEWITHSUNDEEP
pythonenvironment-variablesgoogle-colaboratory
Frying Pan
Frying Pan

Reputation: 407

How do I set environment variables in Google Colab?

On my local computer, I can simply go into "System Properties -> Environment Variables" and add a new variable with its value in user variables.

Then, I can retrieve this value by using this in Python:

import os
os.environ["VAR_NAME"]

However, I just recently started using Google Colab, and it seems it's not able to detect the environment variable, as it gives me this error:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-36-28128554cf91> in <module>()
      1 import os
----> 2 os.environ["REDDIT_NAME"]

/usr/lib/python3.7/os.py in __getitem__(self, key)
    679         except KeyError:
    680             # raise KeyError with the original key value
--> 681             raise KeyError(key) from None
    682         return self.decodevalue(value)
    683 

KeyError: 'REDDIT_NAME'

How should I go about so that Google Colab can detect my user environment variables? Is there a specific path I need to modify?

Thanks.

Upvotes: 20

Views: 25655

Answers (4)

Keiku
Keiku

Reputation: 8803

Install the following packages.

!pip install --quiet openai python-dotenv

Create a .env file like this. Below is an example of an openai key:

/content# cat .env
OPENAI_API_KEY="sk-*"
OPENAI_ORGANIZATION="org-*"
/content#

In Colab, it can be read as an environment variable in the following cell.

import dotenv
dotenv.load_dotenv('./.env')

You can hide it as an environment variable and assign it to the variable.

import os
import openai
openai.api_key = os.environ["OPENAI_API_KEY"]

NOTE: I'm just using openai as an example. If you want to load something else as a variable, change it accordingly.

Upvotes: 2

Mahesh
Mahesh

Reputation: 804

Considering other approaches to set environment variables are mostly via .dotenv file. Here's an simple snippet that I use to load environment variables from Google Colab's Secret.

from google.colab import userdata
import os

openai_key = userdata.get('OPENAI_API_KEY')
os.environ['OPENAI_API_KEY'] = openai_key

Code Explanation:

In the above example, I get the Google Colab's secret OPENAI_API_KEY and assign it to a variable using userdata.get(<colab-secret-name>). The variable is used to set the operating system's environment variables. This way we could easily fetch secret and assign it to env using os.environ['OPENAI_API_KEY'].

Note: Make sure the Notebook has the secret access.

Upvotes: 1

Samudra
Samudra

Reputation: 1233

Google Colab now supports adding keys and secrets on the notebook itself

Source: https://twitter.com/GoogleColab/status/1719798406195867814

Configure your code by storing environment variables, file paths, or keys. Values stored here are private, visible only to you and the notebooks that you select.

Secret name cannot contain spaces.

Steps:

  1. Navigate to the "Secrets" pane from the left navigation bar
  2. Add the secret name and value you want to manage
  3. Turn on the toggle of "Notebook access"
  4. Access your secret keys in Python via:

from google.colab import userdata
userdata.get('secretName')

If you need the secret to be set as an environment variable, then:

%env KAGGLE_USERNAME=abcdefgh

If the value is in a variable you can also use

%env KAGGLE_USERNAME=$username

Upvotes: 17

Phạm Ngọc Qu&#253;
Phạm Ngọc Qu&#253;

Reputation: 329

Like this

import os
os.environ['REDDIT_NAME'] = 'something'
print(os.getenv('REDDIT_NAME'))

Or using dotenv lib. Keep environment in a file:

  import dotenv
  dotenv.load_dotenv(os.path.join(os.path.dirname(__file__), './.env'))

in .env file: REDDIT_NAME = something

and ignore .env file when push to git

Upvotes: 8

Related Questions