Does Python search for a builtin module before searching for a nonbuiltin module?

Question

Specific example:

in /tmp/test, I create a file itertools.py with following content:

def f():
    print("it is my own itertools.")

I know there is a builtin module named itertools

Then I run python3 from /tmp/test,

>>> import itertools
>>> itertools.__dict__
{'_tee': <class 'itertools._tee'>, 'groupby': <class 'itertools.groupby'>, ...}

So I import the builtin module itertools instead of /tmp/test/itertools.py.

It seems that Python searches for a builtin module before searching for a nonbuiltin module. This is contrary to Python modules with identical names (i.e., reusing standard module names in packages). Why?

---

General rules:

From Python in a Nutshell

When a module is loaded, __import__ first checks whether the module is built-in. The tuple sys.builtin_module_names names all built-in modules, but rebinding that tuple does not affect module loading.
The search for built-in modules also looks for modules in platform-specific loca‐ tions, such as the Registry in Windows.

If module M is not built-in, __import__ looks for M ’s code as a file on the filesystem. __import__ looks at the strings, which are the items of list sys.path , in order.

From Learning Python

In many cases, you can rely on the automatic nature of the module import search path and won’t need to configure this path at all. If you want to be able to import user- defined files across directory boundaries, though, you will need to know how the search path works in order to customize it. Roughly, Python’s module search path is composed of the concatenation of these major components, some of which are preset for you and some of which you can tailor to tell Python where to look:

1. The home directory of the program
2. PYTHONPATH directories (if set)
3. Standard library directories
4. The contents of any .pth files (if present)
5. The site-packages home of third-party extensions

Are the five places in Learning Python stored in sys.path?

Are the five places in Learning Python searched only after failing to find a builtin module in sys.builtin_module_names?

Is "3. Standard library directories" not including the builtin modules? Where are the builtin modules stored? What are the relations between "3. Standard library directories" and the builtin modules?

Thanks.

Answer 1

This is only a partial answer but it may help clear up some concepts.

• Builtin modules are typically implemented in C (at least for CPython) and compiled. These are listed in sys.builtin_module_names. Examples of such modules are sys, itertools and _collections (note the leading underscore).

• Then there are the standard library modules. These are normal Python files and on Windows located in a folder lib inside your Python installation. For example collections (without underscore) or copy...

• Then there are installed extension modules. These can be compiled modules, normal Python files, etc. On Windows these are found in the site_packages folder inside the lib folder.

If I look at my sys.path:

['',                                           # current directory
 '...\\Python\\python35.zip',                  # no idea
 '...\\Python\\DLLs',                          # compiled modules (not builtins)
 '...\\Python\\lib',                           # standard library modules
 '...\\Python',                                # Python installation folder
 '...\\Python\\lib\\site-packages',            # installed modules
 ...]

It seems like 1, 3 and 5 are included in my sys.path so maybe 2 and 4 (if set) would be included in there as well. But that could also be something Windows specific.

As for your title question:

Does Python search for a builtin module before searching for a nonbuiltin module?

Yes! The builtins are searched first, before it progresses to look for a module in the current directory or the standard library (or in the installed modules).

For example if you have a sys.py a copy.py file in your current working directory and you try:

>>> import sys   # a builtin module
>>> sys
<module 'sys' (built-in)>     ... not the one from the current working directory

>>> import copy  # a standard library module
>>> copy.__file__
... current working directory ... not the one from the standard library

If you're really interested in the specifics it would probably be a good idea to consult the importlib documentation and the referred PEPs. Personally, I wouldn't go down that path just out of curiosity.