CODEWITHSUNDEEP
pythonyamlpyyaml
Smed
Smed

Reputation: 265

Execute Python code embedded in YAML file

I'm wondering if it's possible to execute Python code that's embedded in a YAML file. For example, my YAML file currently contains a list:

list: [1,2,3,4]

But I'd prefer to enter this list in my YAML file using numpy:

list: np.arange(1,5)

I saw "How to embed Python code into YAML?" describing how to embed the Python code using literal scalar block style, but it doesn't describe how to actually execute the embedded code.

Upvotes: 8

Views: 18064

Answers (3)

bdavis
bdavis

Reputation: 151

Use of PyYAML custom Python tags can accomplish this without additional code.

https://pyyaml.org/wiki/PyYAMLDocumentation#yaml-tags-and-python-types

For the list example in the question:

list: !!python/object/apply:numpy.arange [1,5]

Working Python example:

import yaml; yaml.load("list: !!python/object/apply:numpy.arange [1,5]", yaml.Loader)
Out[0]: {'list': array([1, 2, 3, 4])}

You can leverage this further to do some crazy things with eval from the YAML file. The following code can be used to compile functions that are emitted from the YAML processing. (Eval is normally limited to a single statement)

This example works for me on Windows CPython 3.9 with PyYAML 6.0. (The compiler hacking to pull out the internal code object might break on another version)

import yaml
s = """<below yaml code>"""
y = yaml.load(s, yaml.Loader)
assert y['add_one'](1) == 2
assert not y['is_instance_int']('foo')
assert y['function_result'] == 7

YAML input

# this creates a compiler object that can be used to return a compiled function from string code without using exec
# there is some trickiness because we must pass in the builtins module into global list using __builtins__
compile_to_function: &compiler !!python/object/apply:eval
    - |
        function_type(
           # the inner function code is the first code object attached to the outer code in co_consts
           [c for c in compile(code, '__dynamic__', 'exec').co_consts if isinstance(c,code_type)][0],
           {'function_type': function_type, 'code_type': code_type, **__builtins__, '__builtins__': __builtins__}
        )
    - function_type: !!python/name:types.FunctionType
      code_type: !!python/name:types.CodeType
      code: |
        def compile_to_function(code):
            return function_type([c for c in compile(code, '', 'exec').co_consts if isinstance(c, code_type)][0], __builtins__)


# some examples of compiling things
add_one: &add_one !!python/object/apply:eval
    - compiler(code)
    - compiler: *compiler
      code: |
        def add_one(x):
            x = x + 1
            return x

is_instance_int: !!python/object/apply:eval
    - compiler(code)
    - compiler: *compiler
      # note here that isinstance needs the builtins to be present
      code: |
        def is_instance_int(x):
            return isinstance(x, int)

function_result: !!python/object/apply:eval
    - add_one(1*2*3)
    - add_one: *add_one

Upvotes: 5

PyLover
PyLover

Reputation: 41

For Python 3+ kindly use below code

import yaml
with open("data.yml", "r") as f:
    data = yaml.load(f,Loader=yaml.FullLoader)
exec(data['list'])

yml: data.yml

list: |
    import numpy as np
    print(np.arange(1,5))

Upvotes: 1

Thomasleveil
Thomasleveil

Reputation: 104155

It is possible. The following Python script will do just that provided that your YAML file is named data.yml.

import yaml
with open("data.yml", "r") as f:
    data = yaml.load(f)
exec data['list']

Also make sure to add the imports in your Python code in the YAML file:

list: |
    import numpy as np
    print np.arange(1,5)

Upvotes: 16

Related Questions