CODEWITHSUNDEEP
pythonyaml
eng2019
eng2019

Reputation: 1035

Passing Python functions from YAML file

I have a YAML script where the users can define some parameters. They can also state in the YAML file a custom function they want to apply. For example:

processes:
    args1: 10
    args2: 5
    funct: np.sum

This YAML will be passed to a function like this:

def custom_func(args1, args2, funct):
    return funct(args1, args2)

With the YAML example above, I expect custom_func() to execute np.sum(10,5). How can I make the data in YAML file to be parsed as Callable? eval() probably does the job but it might have security issue. Is there any proper way? Thanks!

Upvotes: 3

Views: 11593

Answers (2)

12944qwerty
12944qwerty

Reputation: 1925

Yes, eval and exec are both bad practices. Good that you know that :D

You can use dictionaries to do this. What you can do is define a dictionary and make the key the funct value in YAML. The value will be an uncalled function. This is similar to a variable.

funct_parser = {
    "np.sum": np.sum # Notice how this doesn't have ()
}

You can then use funct against the dictionary to call the proper method. Something like the following:

def custom_func(args1, args2, funct):
    my_func = funct_parser[str(funct)] # This will cause my_func to be a copy of `np.sum`.
    return my_func(args1, args2) # Call my_func

BTW, you don't really have to do custom_func on multiple lines, I just did so to explain it better.

Alternative choice:
If you don't feel like hardcoding every single part in a dictionary, you can use getattr().

# This code assumes you have numpy imported as np (You can always change it)
def custom_func(args1, args2, funct):
    funct = funct.split("np.")[0] # Will work for np. You can always change to other modules when needed.
    return getattr(np, funct)

Upvotes: 0

JL Peyret
JL Peyret

Reputation: 12214

You can also load the function given its Python dotted path.

However, this only gets the function and calls it, it does not try to figure out calling signature, which is np.sum([5,10]), not np.sum(10,5)

from yaml import safe_load as yload
from importlib import import_module

def load_func(dotpath : str):
    """ load function in module.  function is right-most segment """
    module_, func = dotpath.rsplit(".", maxsplit=1)
    m = import_module(module_)
    return getattr(m, func)
    
yaml = """
processes:
    args1: 10
    args2: 5

    #gotta give real package name, not `np`
    funct: numpy.sum
"""

di = yload(yaml)
pr = di["processes"]
func = load_func(pr["funct"])
array = [pr["args1"], pr["args2"]]

print (f"{func=} {func(array)=}")

output:

func=<function sum at 0x106645e50> func(array)=15

Upvotes: 6

Related Questions