Distinction between Python environment and Jupyter kernel in VS Code

Question

In Python I can create a virtual environment in VS Code with the following commands, I'll also install a kernel to the same virtual environment:

python3 -m venv .venv
source .venv/bin/activate
pip install ipykernel
python3 -m ipykernel install --user --name=.venv

I then go in VS Code and I can Select Kernel and am given three options:

1. Python Environment
2. Jupyter Kernel
3. Existing Jupyter Server

No matter which I choose it just gives me various ways to select the .venv environment I just setup. When I am trying to select a kernel why I am being presented with .venv environments? Aren't kernels and environments two distinct things in Python? I can already operate out of my .venv environment but can't figure out how to actually select the Jupyter kernel I installed into the .venv environment.

When I look at tutorials it seems there's supposed to be the phrase Jupyter Server: Local at the bottom of my VS Code but that's totally missing. I'm not using conda and want to avoid it if possible.

screenshot for reference

Answer 1

Relationship between Environments and Kernels

1. Python Environments

   • These are isolated spaces where you can manage dependencies and packages specific to the project.
   • A virtual environment like .venv provides the base Python interpreter and tools.

2. Jupyter Kernels:

   • Kernels are specific to Jupyter and allow Jupyter Notebooks (or VS Code notebooks) to execute code. A kernel is associated with a Python environment, meaning it uses the interpreter and packages from that environment.

   When you install the ipykernel package in a virtual environment and register it, you're creating a kernel that can be selected in Jupyter (or VS Code) and will use that environment.

Why you see .venv Environments when selecting a Kernel?

The Jupyter extension in VS Code associates environments with kernels because kernels are backed by environments. When you register a kernel using python3 -m ipykernel install, it's tied to the .venv environment you created, so it shows up as an option.

In VS Code, the kernel selection menu shows environments because the kernel you're selecting is tied to a specific Python environment. It's a bit confusing but normal behavior.

Resolving the "Missing Jupyter Server: Local" issue

1. Ensure Jupyter extension is installed

   • Install the Jupyter extension in VS Code.
   • Install the Python extension (this handles Python environments and interpreters).

2. Verify Jupyter is installed in the Virtual Environment

   In your .venv, install Jupyter - pip install jupyter

3. Select the .venv Environment in VS Code

   • Open VS Code.
   • Press Ctrl+Shift+P (or Cmd+Shift+P on macOS).
   • Type "Python: Select Interpreter" and choose the .venv environment you created.

4. Confirm Kernel is registered

   You can check if the kernel is registered by running this in your .venv - jupyter kernelspec list, this will show an entry for .venv.

5. Ensure Jupyter Server is running locally

   • Open a notebook or .ipynb file in VS Code.

   • At the bottom of VS Code, you should see "Jupyter Server: Local". If not,

     • Click the kernel selection dropdown in the top-right corner of the notebook editor.
     • Choose "Python Environment" and select your .venv environment.

6. Test Notebook execution

   • Open or create a .ipynb file in VS Code.
   • Select the kernel associated with your .venv.
   • Run a cell to verify the notebook executes using the correct environment.

Answer 2

This used to annoy me a lot, to the point I developed my own solution for it.

I recommend to use pipenv to manage your virtual environments. This is the set up I have.

My workflow is as follows. From the directory I am starting project in, from the terminal, I run

pipenv shell 

This will create a virtual environment for you and manage its location for you too.

I then install jupyter-notebook,Ipython and some other jupyter-related packages to that specific virtual environment using a script I have developed over my time in industry.

#!/usr/bin/env python

import subprocess
import os

print(f"Current working dirctory: { os.getcwd() }")
pipenv_name = os.environ['VIRTUAL_ENV'].split("/")[-1]
print(f"Installing ipykernel to environment: {pipenv_name}")
python_loc = subprocess.check_output(["which", "python"]).decode("utf-8").rstrip("\n")
pip_loc = subprocess.check_output(["which", "pip"]).decode("utf-8").rstrip("\n")
print("python location: ", python_loc)
print("pip location: ", pip_loc)
print("Install ipykernel to user in environment")
try:
    subprocess.check_call([pip_loc,"install","ipykernel", "jupyterlab", "jupyter_nbextensions_configurator", "jupyter-http-over-ws", "ipywidgets", "widgetsnbextension"])
except subprocess.CalledProcessError:
    subprocess.check_call(["pip","install", "ipykernel", "jupyterlab", "jupyter_nbextensions_configurator", "jupyter-http-over-ws",  "ipywidgets", "widgetsnbextension"])
    pass
try:
    subprocess.check_call([str(python_loc), "-m", "ipykernel", "install", "--user", f"--name={pipenv_name}"])
except subprocess.CalledProcessError:
    subprocess.check_call(["python3","-m","ipykernel","install","--user",f"--name={pipenv_name}"])
    pass

I keep this script in my home directory under ~/.install-jpnb.py and from within my active virtual environment run

~/.install-jpnb.py

Make sure you execute the above script , ~/.install-jpnb.py, from the terminal inside your virtual environment, before launching vscode

Then, you can open up vscode, and there will be a choice of virtual environments in which you can run jupyter-notebooks (depending on how many you've previously created under this workflow)

Like in my example screenshot, I have two virtual environments both with all the jupyter-notebook packages install, which I can switch between from just within vscode.

Once you delete the virtual environment using something like

pipenv --rm

you're jupyter-notebook environment associated to that virtual environment will also be removed.

Note I only used pipenv to manage the environment. I dont use its requirements Pipfile file or any other features it has, although you are free to explore them. I still just used pip freeze > requirements.txt to generate the installation requirements.

But nethertheless, pipenv allowed me to easily work with jupyter-notebooks across different environments easily.