Difference between an ipynb kernal (Jupyter lab / Jupyter notebook) and a python virtual environment ( created through venv)

Question

I wanted to ask about the difference between a Jupyter notebook kernel and a Python virtual environment created through venv. From what I understand:

1. Jupyter notebook kernels are used for notebooks, while Python virtual environments are used for isolation.
2. What prompted this question was that I downloaded Jupyter Notebook in one virtual environment and created a kernel for it. Later, I wanted to switch to another virtual environment, which confused me. Can this be done, or do I need to install Jupyter Notebook in the new virtual environment and launch it from there?

I made two different kernals for two different python environments so does this mean both kernals are like python environments in the jupyter lab or not.

Answer 1

TL;DR

• A Jupyter Kernel is a pipeline to executing code via Python/IPython, but not a venv itself.
• Create a new Jupyter Kernelspec pointing to the other venv Python and add it to your current Jupyter Kernel to add that venv. See https://stackoverflow.com/a/44072803/22692815 for more info.

If you want to switch the venv of your Jupyter server, you have two choices. But first, some technical details:

What is a Jupyter Kernel?

What is a kernel in Jupyter Notebook and how it is different/similar to actual kernel (related to Operating System)?

Here's an analogy for this. Your Python interpreter is like a soldier on the battlefield. It is the one directly fighting the enemy, and follows orders from command. Your Jupyter Kernel is the phone that connects command to the soldier - it makes the soldier do things, but isn't a virtual environment (or soldier) itself. It is not independent, but makes it so command (your code) can control the soldier (the python interpreter). Basically:

• Soldier = Python (on battlefield, can execute commands)
• Phone/Command Line = Kernel (communicates with soldier, pipes commands)
• Commander = Your code - passed to "soldier" via the Kernel

Relaunch from the other venv

This one is easy, and it means that the default Jupyter Kernel on the new instance you launch will use the new venv. However, for the sake of this question, let's assume you don't want to do that, and instead want to be able to use this venv in your current Jupyter server.

Add a new Kernel

https://stackoverflow.com/a/44072803/22692815 Here are the steps you need to add a new kernel.

1. Create a new directory in your project named jupyter_kernels.
2. cd into the directory, then touch new_kernel.json.
3. Add the following spec to new_kernel.json, adjusting for your venv path and desired name:

{
 "argv": [
  "/your/other/venv/bin/python3.5",
  "-m",
  "ipykernel",
  "-f",
  "{connection_file}"
 ],
 "language": "python",
 "display_name": "Your Other Kernel Name"
}

4. Save the file.
5. Run jupyter kernelspec install jupyter_kernels or wherever the new file you created is.
6. Reload the Jupyter Notebook page - then select the new kernel from the top right.