CODEWITHSUNDEEP
pythongoogle-cloud-platformgoogle-cloud-vertex-ai
zabop
zabop

Reputation: 7912

How to update Python on Vertex AI notebooks?

I am working in notebooks provided in the Workbench section of Vertex AI. I need an updated version of Python, but I only have access to Python 3.7 in these notebooks. I have successfully followed these steps and if I run python3.8 --version in terminal, I get Python 3.8.2, which is good, but python --version still returns Python 3.7.12. If, following this answer and restarting notebook's kernel, I run

from platform import python_version
print(python_version())

in a notebook, and I get 3.7.12.

How do I get a notebook in Vertex AI supporting an up-to-date Python version?

Upvotes: 17

Views: 11099

Answers (4)

Brian Bien
Brian Bien

Reputation: 763

Python 3.10 is now installed by default on the latest image: https://cloud.google.com/deep-learning-vm/docs/release-notes

Upvotes: 2

Gilles Charlier
Gilles Charlier

Reputation: 141

For me, @ewertonvsilva's answer wasn't working. I had to install the ipython kernel at user level then deactivate the kernel before refreshing the page to make the environment appearing on Jupyter Lab, All commands combined :

# create a new conda env:
$ conda create -n python38 python=3.8

# Activate your new Python 3.8 environment:
$ conda activate python38

#install ipykernel when logged in the new env:
(python38)$ conda install ipykernel

# install the ipython kernel at user level    
(python38)$ ipython kernel install --user --name=python38

# Deactivate the new environment
(python38)$ conda deactivate

Then refresh the page.

Sources :

How to add conda environment to jupyter lab

Installing the IPython kernel - IPython Documentation

Upvotes: 14

Joseph Lee
Joseph Lee

Reputation: 126

After a few trials and errors, I found that:

  • Changes under /opt/conda will not be preserved when you change/stop instances.
  • Custom conda env needs to be saved to $HOME.
  • conda activate $ENV will not work, since conda init was not executed.
  • pip install installs to /opt/conda/lib/python3.7/site-packages, which will not be preserved.
  • pip install --user installs to ~/.local/lib/python3.7, which will be applied to every python3.7 kernel, kinda messy.
  • In notebook, !pip uses pip in shell, not in current kernel.
  • In notebook, %pip uses pip in current kernel.
  • In custom env, don't use --user flag, or it installs to ~/.local/lib.

Run the commands in system terminal:

VENV=new_env

# create new env in `$HOME`
conda create -y -q -p $HOME/conda_env/$VENV python=3.8 ipykernel

# activate env
source /opt/conda/bin/activate ~/conda_env/$VENV

# register kernel to `$HOME/.local/share/jupyter/kernels`, so it will be preserved
python -m ipykernel install --user --name $VENV

# install your packages, WITHOUT `--user`
pip install numpy==1.22

# check package installation path
pip list -v

Now you can change kernel in Launcher (takes a few minutes to refresh), or the box on the top right of notebook. The kernel will be preserved.

In notebook with custom env:

# use `%pip` in notebook, instead of `!pip`
%pip install numpy==1.22  # `/home/jupyter/conda_env/$VENV/lib/python3.8/site-packages`
!pip install numpy==1.22  # `/opt/conda/lib/python3.7/site-packages`
!pip install --user numpy==1.22  # `~/.local/lib/python3.7/site-packages`

# you may see the difference in custom env
%pip list -v
!pip list -v

Upvotes: 9

ewertonvsilva
ewertonvsilva

Reputation: 1945

#create a new conda env:
$ conda create -n python38 python=3.8

#Activate your new Python 3.8 environment:
$ conda activate python38

#install ipykernel when logged in the new env:
(python38)$ conda install ipykernel

Refresh the page and the new python38 env will be avaiable: enter image description here enter image description here

Upvotes: 13

Related Questions