CODEWITHSUNDEEP
pythonpython-3.ximportmodulepython-import
Ericson Willians
Ericson Willians

Reputation: 7855

Importing dynamically all modules from a folder

I know there is this question, but not only they're not working, it's not exactly what I want. I'm developing a racing game and want to load all tracks from a folder dynamically (They're stored as .py instead of .json). I don't want to know the names of the tracks, since users can mod/add them at will. I just want to import their data. So, for example:

>tracks 
>>track0.py
>>track1.py
>>track2.py
>>track3.py
>>track4.py

Inside each track, I have data like this:

track_ground_data = [
    [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
    [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
    [1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1],
    [1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1],
    [1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1],
    [1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1],
    [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
    [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
]

I need to import each track module like this:

loaded_tracks = [t for t in tracks] # Where tracks is the folder.

And then access a given track_ground_data like this:

loaded_tracks[0].track_ground_data

If I knew Python was going to be so harsh with its imports, I'd have used json instead .py.

Upvotes: 4

Views: 10438

Answers (3)

Bakuriu
Bakuriu

Reputation: 102039

Python does not automatically import submodules contained in a package. Hence import tracks only loads tracks/__init__.py.

However you can put code inside the __init__.py file that imports all the modules it finds in that directory.

For example putting something like this in the __init__.py:

import os
import importlib

for file in os.listdir(os.path.dirname(__file__)):
    mod_name = file.removesuffix(".py")
    if mod_name in ("__init__", "__pycache__"): continue
    importlib.import_module('.' + mod_name, package=__name__)

Should make your submodules available as tracks.trackX when importing only tracks.

Or you could use exec:

import os
import importlib

for file in os.listdir(os.path.dirname(__file__)):
    mod_name = file.removesuffix(".py")
    if mod_name in ("__init__", "__pycache__"): continue
    exec('import .' + mod_name)

A cleaner approach would be to use import hooks or implement your own custom module importer. There are multiple ways to do this using importlib see also sys.path_hooks

Upvotes: 7

Ericson Willians
Ericson Willians

Reputation: 7855

Just for the sake of the good wellfare of future pythoners, I'm posting how I've solved. A friend helped me through it. Coudln't make Bakuriu's solution work, because the modules came empty. Inside __init__.py I've put:

import os

dir = os.path.dirname(os.path.abspath(__file__))
modules = [os.path.splitext(_file)[0] for _file in os.listdir(dir) if not _file.startswith('__')]

tracks = []
for mod in modules:
    exec('from tracks import {}; tracks.append({})'.format(mod, mod))

And then, on the main file, I've loaded it as:

dir = os.path.dirname(os.path.abspath(__file__))
sys.path.append(dir)

from tracks import tracks

And then:

loaded_tracks = [t for t in tracks]

That actually solved it quite well. I was almost switching to JSON / giving up.

Upvotes: 2

Guilherme Marthe
Guilherme Marthe

Reputation: 1124

The problem of dynamically importing modules is faced usually when frameworks have a plug-in or add on system for the community to contribute. Each plug-in or add-on is a module containing classes and functions compliant with the framework's architecture and api.

With that in mind, the solution for "joining the dots" between the framework code and arbitrarily many add-ons is through the importlib present in the python standard library. You seem to face the same structural problem.

Here is a stackoverflow question that was answered with importlib. And the documentation.

Upvotes: 1

Related Questions