CODEWITHSUNDEEP
pythonpython-importspyder
Prokop Hapala
Prokop Hapala

Reputation: 2444

recursive reload() "from some_module import *" in python

I have module called KosmoSuite initialized by folowing __init__.py

...
from chemical_fuels import *
from orbital_mechanics import *
from termodynamics import *
from rail_guns import *
...

in files chemical_fuels.py, orbital_mechanics.py, termodynamics.py, rail_guns.py are some data tables, constants, and functions to conduct some physical computations (for example function orbital_mechanics.escape_velocity(M,R) to compute escape velocity from a planet of given mass and radius)

I would like to use python interpreter as interactive calculator of space-related problems.

However, The problem is interactive developlopment and debugging. When I do

>>> import KosmoSuite as ks
# ... modify something in orbital_mechanics.escape_velocity( ) ...
>>> reload(ks)
>>> ks.orbital_velocity( R, M)

However the ks.orbital_velocity( R, M) is not affected by my modifications to orbital_mechanics.escape_velocity(). Is there some alternative to reload(ks) which does the job ( i.e. reloading of all object, constants and functions imported by from some_module import * recursively )

Even better would be something like 

>>> from KosmoSuite import *
# ... modify something in orbital_mechanics.escape_velocity( ) ...
>>> from KosmoSuite reimport *
>>> orbital_velocity( R, M)

footnote : I'm using Spyder ( Python(x,y) ) right now, but the same is true for default python interpreter. In this question is something about deep-reload ( dreload ) in IPython. I'm not sure if it does exactly this job, but I don't like IPython anyway.

Upvotes: 4

Views: 1774

Answers (3)

thodnev
thodnev

Reputation: 1612

Yes, as long as Python supports metaprogramming, this could be done.

Here's my variant of a function performing such task (Python3):

import importlib, sys

def reload_all(top_module, max_depth=20):
    '''
    A reload function, which recursively traverses through
    all submodules of top_module and reloads them from most-
    nested to least-nested. Only modules containing __file__
    attribute could be reloaded.

    Returns a dict of not reloaded(due to errors) modules:
      key = module, value = exception
    Optional attribute max_depth defines maximum recursion
    limit to avoid infinite loops while tracing
    '''
    module_type = type(importlib)   # get the 'module' type
    for_reload = dict() # modules to reload: K=module, V=depth

    def trace_reload(module, depth):    # recursive
        nonlocal for_reload
        depth += 1
        if type(module) == module_type and depth < max_depth:
            # if module is deeper and could be reloaded
            if (for_reload.get(module, 0) < depth
                and hasattr(module, '__file__') ):
                    for_reload[module] = depth
            # trace through all attributes recursively       
            for name, attr in module.__dict__.items():
                trace_reload(attr, depth)


    trace_reload(top_module, 0)         # start tracing
    reload_list = sorted(for_reload, reverse=True,
                         key=lambda k:for_reload[k])
    not_reloaded = dict()
    for module in reload_list:
        try:
            importlib.reload(module)
        except:     # catch and write all errors
            not_reloaded[module]=sys.exc_info()[0]

    return not_reloaded

It's enough self-documented. If you have some ideas about improvements could be done, here's github project: https://github.com/thodnev/reload_all

Upvotes: 0

otus
otus

Reputation: 5732

A very heavy handed solution, so to speak, is to save the state of sys.modules before you import the module chain, then restore it to its original state before importing the modules again.

import sys
bak_modules = sys.modules.copy()
import KosmoSuite as ks
# do stuff with ks
# edit KosmoSuite
for k in sys.modules.keys():
    if not k in bak_modules:
        del sys.modules[k]
import KosmoSuite as ks

However, there are some caveats:

  1. You may need to reimport even unrelated modules that you've imported in the meanwhile.
  2. If you have created any objects using the old version of the module, they retain their old classes.

Still, I've used it when developing a module while testing it in an interactive session, and if you take the limitations into account, for the most part it works fine.

Upvotes: 3

Ned Batchelder
Ned Batchelder

Reputation: 375764

Reloading modules in Python doesn't work the way you want. Once you create an object ks, it has a reference to the class, and therefore the code. If you reload a module, it will define a new class with the same name as the original. The object still refers to the original class, not the new class.

You might be able to change the class of your existing objects, but if they refer to other objects, then you have to try to change those classes, etc. You'd be fighting the class and module system, trying to re-implement a significant part of the reloading yourself.

You're much better off finding a workflow that works with the way Python modules already behave. IPython notebook lets you experiment interactively, but captures the code so that it can be re-run from the top. There might be other solutions too.

Upvotes: 2

Related Questions