CODEWITHSUNDEEP
python
Dave
Dave

Reputation: 454

Using exec to load module variables, using a file path

A textbook suggests I should be able to do this:

d = {}
exec("C://Users//Dave//Desktop//Bot//bot_config_data.py", globals(), d)
File "<string>", line 1   C://Users//Dave//Desktop//Bot//bot_config_data.py
                           ^
SyntaxError: invalid syntax

I can do this: 

d = {}
exec('from bot_config_data import price_data', globals(), d)

But, I would like to do the former.

I'm trying to write a method which overrides config data from various files.

Am I completely off base here?

Update The book was quite misleading. It posted part of the code, the result of the complete code block, and then gave the remainder. As I was referencing, rather than working through it cover to cover, I tripped over myself.

This is the code I now have:

data = {}
file = 'C:\\Users\\Dave\\Desktop\\Bot\\bot_config_data.py'
with open(file) as f:
    code = compile(f.read(), file, "exec")
    exec(code, globals(), data)
price_data = data["price_data"]

Update2 Using Mad Physicist's answer below, my code would be:

from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader

filepath = 'C:\\Users\\Dave\\Desktop\\Bot\\bot_config_data.py'
module_data = os.path.basename(filepath)
spec = spec_from_loader(module_data, SourceFileLoader(module_data, filepath))
bot_config_data = module_from_spec(spec)
spec.loader.exec_module(bot_config_data)
price_data = bot_config_data.price_data

Upvotes: 1

Views: 1826

Answers (2)

Naysayer
Naysayer

Reputation: 1

If you need to import using a dynamic name instead of a hard-coded import statement, you can use importlib.import_module:

price_data = importlib.import_module('bot_config_data').price_data

Huh? How is that "dynamic"? 'bot_config_data' is literally a string literal.

If you really want to import modules dynamically, contrary to what is widely believed, the only way to do this is with exec(). I challenge anyone to find or submit an example of importlib.import_module or __import__ being used to truly dynamically import modules.

To be clear: __import__ allows for the dynamic handling of imports, which is not the same as dynamically importing, as that is a chicken/egg type problem only exec() can solve.

The problem is that importlib.import_module() returns an object, which needs an assignment. (It doesn't just add to the global namespace as is often implied.) But you can't use the same argument on both sides of the assignment. So while it allows you to do handle imports in a more dynamic way, it provides no way of dynamically importing modules.

Upvotes: 0

Mad Physicist
Mad Physicist

Reputation: 114330

You are completely off-base and the textbook is giving you some very bad advice.

exec runs Python code. As in Python statements, not the name of a file. From the docs:

exec(object[, globals[, locals]])

This function supports dynamic execution of Python code. object must be either a string or a code object. If it is a string, the string is parsed as a suite of Python statements which is then executed (unless a syntax error occurs)

That is why your second statement works fine. If you want to run a Python module, import it. If you need to import using a dynamic name instead of a hard-coded import statement, you can use importlib.import_module:

price_data = importlib.import_module('bot_config_data').price_data

This will run the whole import machinery for you, including ensuring that bot_config_data ends up in sys.modules.

If you really need something even fancier, you can use the __import__ machinery. __import__ is the under-the-hood implementation of the import statement:

d = {}
bot_config_data = __import__('bot_config_data', locals=d, from_list=['price_data'])
price_data = bot_config_data.price_data

If you want full control over the process, you can use the low-level machinery described here: How to import a module given the full path?. In particular see my answer to that question, since it describes how to load random text files as Python scripts: https://stackoverflow.com/a/43602557/2988730:

from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader 

spec = spec_from_loader("bot_config_data", SourceFileLoader("bot_config_data", "C:/Users/Dave/Desktop/Bot/bot_config_data.py"))
bot_config_data = module_from_spec(spec)
spec.loader.exec_module(bot_config_data)
price_data = bot_config_data.price_data

Final note: It looks like you blindly converted all the \\ to // in your path. Forward slashes do not need to be escaped, so you only need single forward slashes. If you want to avoid escaping your backslashes, put r in front of your string to make it a raw string.

Upvotes: 2

Related Questions