CODEWITHSUNDEEP
pythonpython-3.ximportjupyter-notebookanaconda
Vedant Modi
Vedant Modi

Reputation: 125

Import local python file to Jupyter Notebook

I need to import a few python files stored in my local memory as modules in my Jupyter Notebook.

My jupyter notebook is store in C:\snip\Pictures\demo and I need to import python files stored in C:\snip\Pictures\demo\mrcnn.

When I try this:

from mrcnn import utils
from mrcnn import visualize
from mrcnn.visualize import display_images
from mrcnn.visualize import display_instances
import mrcnn.model as modellib
from mrcnn.model import log
from mrcnn.config import Config
from mrcnn import model as modellib, utils

I get this error:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-10-fdc1339465a7> in <module>
     17 import matplotlib.image as mpimg
     18 
---> 19 from mrcnn import utils
     20 from mrcnn import visualize
     21 from mrcnn.visualize import display_images

ModuleNotFoundError: No module named 'mrcnn'

mrcnn is the folder name, and utils, visualize... are python files within the folder. How do I solve this?

Upvotes: 0

Views: 15523

Answers (4)

Sau001
Sau001

Reputation: 1674

I am using Jupyter notebooks from VS Code and I faced a similar challenge. I adopted the approach outlined below. I am on Python 3.9.7

Folder structure

I would like to explain my solution using the simplified folder structure shown below. I want to re-use the code in mymodule.py in my notebooks mynotebook1.ipynb.

For this example, lets assume that my sources are located at c:\work which is designated as root in the following diagram.

root(c:\work)
    |
    |----mynotebook1.ipynb
    |
    |
    |----mynotebook2.ipynb
    |
    |----modules
    |    |
    |    |-----__init__.py
    |    |
    |    |-----mymodule.py
    |
    |----notebooks
    |    |
    |    |-----another_notebook.py
    |

Notice that I have notebooks at the top level as well as under the notebooks folder. This is to hilight issues with relative imports.

Content of mymodule.py

class MyAlgorithms(object):
    @staticmethod
    def some_implementation_1():
        print("Inside implmentation 1" )

    @staticmethod
    def some_implementation_2():
        print("Inside implmentation 2" )

Content of init.py

Notice that I am doing a relative import.

from .mymodule import  MyAlgorithms


__all__=["MyAlgorithms"]

Why setting PYTHONPATH is important ?

We will need to set the PYTHONPATH environment variable to the source root, which is c:\work\ in this example.

This is an important step. By doing this, we are making Jupyter's Python kernel aware about our folder structure and this makes discovery of modules easier. If not, we end up in nasty sys.path manipulations within the notebook - which I want to avoid.

Why not use relative imports?

Relative imports works if your notebook is at the top level and modules in child folder. However, if your project structure is such that notebooks is organized in siblings folder, as in the example of another_notebook.ipynb in the notebooks folder, you are likely to see this error:

enter image description here

Setting PYTHONPATH variable before launching Jupyter

If using PowerShell

You could type $env:PYTHONPATH="c:\work\" or the following simplified command if your working folder is already in c:\work\

$env:PYTHONPATH=$PWD.Path

If using Windows CMD

You could type set PYTHONPATH=c:\work or the following simplified command if your working folder is already in c:\work\

set PYTHONPATH=%cd%

Referencing the code of mymodule.py in the notebooks

Once the PYTHONPATH variable is set to the project root , the following syntax works for any notebook - regardless of the folder hierarchy.

import modules
modules.MyAlgorithms.some_implementation_1()

Advantages of this approach

  • It lets me keep more complex code outside of the Jupyter notebook in well organized classes
  • I get better GIT control and change tracking than that what I would have if all the code were in the notebook
  • Setting PYTHONPATH to the project root significantly improves the developer experience when it comes to importing local modules

Disadvantages

  • I have to re-start the Jupyter notebook (main menu-->Kernel --> Restart kernel) whenever I make some changes to the code in the modules folder.

Update on 12 th April 2024

  • Emphasis on PYTHONPATH environment variable
  • Typo in Python class due to indentations
  • Changed to relative import in __init__.py

Update on 13 th April 2024

  • Explanation on why setting PYTHONPATH simplifies imports

enter image description here

Upvotes: 2

SiliconXu
SiliconXu

Reputation: 116

This might be outdated. I came across this post as I have the same issue. And when I saw Vedant Modi's comment saying that "Made a new notebook and it worked, no idea why", it comes to me that it might works to try to restart the kernel. And yes, restart the kernel fix the problem.

Upvotes: 1

PaoloTCS
PaoloTCS

Reputation: 21

What you are doing works with using .py files, but Jupyter files are a bit different. You need to import modules by adding %run If you wish to also use the variables in the py file, use %run -i.

Upvotes: 2

elaaf
elaaf

Reputation: 81

Do this. Print the sys.path like this. This lists the directories that the python interpreter is looking inside when doing import.

import sys

print(sys.path)

Output (Something like this):

['/mnt/c/Users/lapto/Desktop/demos', '/home/elaaf/anaconda3/lib/python38.zip', '/home/elaaf/anaconda3/lib/python3.8', '/home/elaaf/anaconda3/lib/python3.8/lib-dynload', '/home/elaaf/anaconda3/lib/python3.8/site-packages', '/home/elaaf/anaconda3/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg']

If your current working directory is in the output list. Your import calls are not simply not looking in the current directory.

Try adding the current directory to the sys.path and importing the modules

import sys

sys.path.append(".")

Let me know if this helps.

Upvotes: 1

Related Questions