CODEWITHSUNDEEP
pythonmodule
JuanP. Zuniga
JuanP. Zuniga

Reputation: 67

Call a function for every script inside a folder

Is there a way (using only python. i.e.: without a bash script nor another language code) to call a specific function in every script inside a folder without needing to import all of them explicitly.

For example, let's say that this is my structure:

main.py
modules/
    module1.py
    module2.py
    module3.py
    module4.py

and every moduleX.py has this code:

import os

def generic_function(caller):
    print('{} was called by {}'.format(os.path.basename(__file__), caller))

def internal_function():
    print('ERROR: Someone called an internal function')

while main.py has this code:

import modules
import os

for module in modules.some_magic_function():
    module.generic_function(os.path.basename(__file__))

So if I run main.py, I should get this output:

module1.py was called by main.py
module2.py was called by main.py
module3.py was called by main.py
module4.py was called by main.py

*Please note that internal_function() shouldn't be called (unlike this question). Also, I don't want to declare explicitly every module file even on a __init__.py

By the way, I don't mind to use classes for this. In fact it could be even better.

Upvotes: 0

Views: 769

Answers (2)

JuanP. Zuniga
JuanP. Zuniga

Reputation: 67

While sophros' approach is quickly and enough for implicitly importing the modules, you could have issues related to controlling every module or with complex calls (like having conditions for each calls). So I went with another approeach:

First I created a class with the function(s) (now methods) declared. With this I can avoid checking if the method exists as I can use the default one if I didn't declare it:

# main.py
class BaseModule:
    def __init__(self):
        # Any code
    
    def generic_function(self, caller):
        # This could be a Print (or default return value) or an Exception
        raise Exception('generic_function wasn\'t overridden or it was used with super')

Then I created another class that extends the BaseModule. Sadly I wasn't able to get a good way for checking inherence without knowing the name of the child class so I used the same name for every module:

# modules/moduleX.py
from main import BaseModule

class GenericModule(BaseModule):
    def __init__(self):
        BaseModule.__init__(self)
        # Any code
    
    def generic_function(self, caller):
        print('{} was called by {}'.format(os.path.basename(__file__), caller))

Finally, in my main.py, I used the importlib for importing the modules dynamically and saving an instance for each one, so I can use them later (for sake of simplicity I didn't save them in the following code, but it's easy as using a list and appending every instance on it):

# main.py
import importlib
import os

if __name__ == '__main__':
    relPath = 'modules' # This has to be relative to the working directory

    for pyFile in os.listdir('./' + relPath):
        # just load python (.py) files except for __init__.py or similars
        if pyFile.endswith('.py') and not pyFile.startswith('__'):
            # each module has to be loaded with dots instead of slashes in the path and without the extension. Also, modules folder must have a __init___.py file
            module = importlib.import_module('{}.{}'.format(relPath, pyFile[:-3]))
            # we have to test if there is actually a class defined in the module. This was extracted from [1]
            try:
                moduleInstance = module.GenericModule(self)
                moduleInstance.generic_function(os.path.basename(__file__)) # You can actually do whatever you want here. You can save the moduleInstance in a list and call the function (method) later, or save its return value.
            except (AttributeError) as e:
                # NOTE: This will be fired if there is ANY AttributeError exception, including those that are related to a typo, so you should print or raise something here for diagnosting
                print('WARN:', pyFile, 'doesn\'t has GenericModule class or there was a typo in its content')

References:

[1] Check for class existence

[2] Import module dynamically

[3] Method Overriding in Python

Upvotes: 0

sophros
sophros

Reputation: 16728

You can use exec or eval to do that. So it would go roughly this way (for exec):

def magic_execute():
    import os
    import glob
    for pyfl in glob.glob(os.path(MYPATH, '*.py'):
        with open(pyfl, 'rt') as fh:
            pycode = fh.read()
            pycode += '\ngeneric_function({})'.format(__file__)
            exec(pycode)

The assumption here is that you are not going to import the modules at all.

Please note, that there are numerous security issues related to using exec in such a non-restricted manner. You can increase security a bit.

Upvotes: 1

Related Questions