CODEWITHSUNDEEP
pythonmoduleexecconfiguration-files
rtrwalker
rtrwalker

Reputation: 1021

How to use exec safely? I would like to use a dynamically-loaded code module as a configuration file

After reading the Software Carpentry essay on Handling Configuration Files I'm interested in their Method #5: put parameters in a dynamically-loaded code module. Basically I want the power to do calculations within my input files to create my variables.

Based on this SO answer for how to import a string as a module I've written the following function to import a string or oen fileobject or STringIO as a module. I can then access varibales using the . operator:

import imp

def make_module_from_text(reader):
    """make a module from file,StringIO, text etc

    Parameters
    ----------
    reader : file_like object
        object to get text from

    Returns
    -------
    m: module
        text as module

    """
    #for making module out of strings/files see https://stackoverflow.com/a/7548190/2530083    

    mymodule = imp.new_module('mymodule') #may need to randomise the name; not sure
    exec reader in mymodule.__dict__    
    return mymodule

then

import textwrap
reader = textwrap.dedent("""\
    import numpy as np

    a = np.array([0,4,6,7], dtype=float)
    a_normalise = a/a[-1]    
    """)

mymod = make_module_from_text(reader)
print(mymod.a_normalise)

gives

[ 0.          0.57142857  0.85714286  1.        ]

All well and good so far, but having looked around it seems using python eval and exec introduces security holes if I don't trust the input. A common response is "Never use eval orexec; they are evil", but I really like the power and flexibility of executing the code. Using {'__builtins__': None} I don't think will work for me as I will want to import other modules (e.g. import numpy as np in my above code). A number of people (e.g. here) suggest using the ast module but I am not at all clear on how to use it(can ast be used with exec?). Is there simple ways to whitelist/allow specific functionality (e.g. here)? Is there simple ways to blacklist/disallow specific functionality? Is there a magic way to say execute this but don't do anythinh nasty.

Basically what are the options for making sure exec doesn't run any nasty malicious code?

EDIT:

My example above of normalising an array within my input/configuration file is perhaps a bit simplistic as to what computations I would want to perform within my input/configuration file (I could easily write a method/function in my program to do that). But say my program calculates a property at various times. The user needs to specify the times in some way. Should I only accept a list of explicit time values so the user has to do some calculations before preparing the input file? (note: even using a list as configuration variable is not trivial see here). I think that is very limiting. Should I allow start-end-step values and then use numpy.linspace within my program? I think that is limiting too; whatif I want to use numpy.logspace instead? What if I have some function that can accept a list of important time values and then nicely fills in other times to get well spaced time values. Wouldn't it be good for the user to be able to import that function and use it? What if I want to input a list of user defined objects? The thing is, I don't want to code for all these specific cases when the functinality of python is already there for me and my user to use. Once I accept that I do indead want the power and functionality of executing code in my input/configuration file I wonder if there is actually any difference, security wise, in using exec vs using importlib vs imp.load_source and so on. To me there is the limited standard configparser or the all powerful, all dangerous exec. I just wish there was some middle ground with which I could say 'execute this... without stuffing up my computer'.

Upvotes: 2

Views: 1611

Answers (2)

DaveP
DaveP

Reputation: 7102

If you are willing to use PyPy, then its sandboxing feature is specifically designed for running untrusted code, so it may be useful in your case. Note that there are some issues with CPython interoperability mentioned that you may need to check.

Additionally, there is a link on this page to an abandoned project called pysandbox, explaining the problems with sandboxing directly within python.

Upvotes: 0

Armin Rigo
Armin Rigo

Reputation: 12900

"Never use eval or exec; they are evil". This is the only answer that works here, I think. There is no fully safe way to use exec/eval on an untrusted string string or file.

The best you can do is to come up with your own language, and either interpret it yourself or turn it into safe Python code before handling it to exec. Be careful to proceed from the ground up --- if you allow the whole Python language minus specific things you thought of as dangerous, it would never be really safe.

For example, you can use the ast module if you want Python-like syntax; and then write a small custom ast interpreter that only recognizes a small subset of all possible nodes. That's the safest solution.

Upvotes: 3

Related Questions