CODEWITHSUNDEEP
pythonpytest
Will Ayd
Will Ayd

Reputation: 7164

How to mark specific combinations of parametrized pytest arguments?

According to the pytest documentation, I can generate combinations of multiple parametrized arguments as follows:

@pytest.mark.parametrize("x", [0, 1])
@pytest.mark.parametrize("y", [2, 3])
def test_foo(x, y):
    pass

I can also apply marks to individual parameters as such:

@pytest.mark.parametrize("test_input,expected", [
("3+5", 8),
("2+4", 6),
pytest.param("6*9", 42,
             marks=pytest.mark.xfail),
])
def test_eval(test_input, expected):
    assert eval(test_input) == expected

Is there a reasonable way to combine the two methodologies and apply a mark to a particular combination of parameters? For instance, can I apply a pytest.mark.xfail ONLY to the test_foo instance that gets generated with x==0 and y==2?

Upvotes: 8

Views: 3465

Answers (3)

Yinon Ehrlich
Yinon Ehrlich

Reputation: 626

pytest.param does the trick.

Look for It is also possible to mark individual test instances within parametrize in the link above.

Upvotes: 0

mmore500
mmore500

Reputation: 101

A simpler solution that covers many use cases is to mark each component with a "partial tag" and then filter by tests that contain all partial tags.

For example, mark one case tag_2a and tag_2b then filter for -m tag_2a and tag_2b or -m (not tag_2a and not tag_2b).

For xfail you could possibly use a hook via conf.py to automatically mark tests containing both tag_2a and tag_2b as xfail.

Upvotes: 0

s-m-e
s-m-e

Reputation: 3729

The approach I prefer is to generate my arguments through a simple helper function. Have a look at the following example:

import pytest

def __get_param_xy__(x = [], y = [], xfail = []): # ugly demonstrator ...
    out_list = []
    for xx in x: # this could be a clever list comprehension ...
        for yy in y: # this one, too ...
            out_tup = (xx, yy)
            if out_tup in xfail: # the ones you expect to fail
                out_tup = pytest.param(*out_tup, marks = pytest.mark.xfail)
            out_list.append(out_tup)
    return out_list

@pytest.mark.parametrize('x,y', __get_param_xy__(
    x = [0, 1],
    y = [2, 3],
    xfail = [(0, 2)]
    ))
def test_foo(x, y):
    assert not (x == 0 and y == 2)

It still uses a single parametrize decorator, but it comes fairly close to what you want and is easy to read and understand.

EDIT (1): You can actually implement the helper function as a generator. The following works just fine:

def __get_param_xy__(x = [], y = [], xfail = []): # ugly generator ...
    for xx in x: # this could be a clever list comprehension ...
        for yy in y: # this one, too ...
            out_tup = (xx, yy)
            if out_tup in xfail: # the ones you expect to fail
                out_tup = pytest.param(*out_tup, marks = pytest.mark.xfail)
            yield out_tup

EDIT (2): Since it has been asked in the comments, this can actually be generalized for an arbitrary number of parameters and does not conflict with fixtures. Check the following example:

import pytest

class __mock_fixture_class__:
    def __init__(self):
        self.vector = []
    def do_something(self, parameter):
        assert parameter != (0, 2)
        self.vector.append(parameter)
    def fin(self):
        self.vector.clear()

@pytest.fixture(scope = 'function')
def mock_fixture(request):
    mock_fixture_object = __mock_fixture_class__()
    def __finalizer__():
        mock_fixture_object.fin()
    request.addfinalizer(__finalizer__)
    return mock_fixture_object

def __get_param_general_generator__(*_, **kwargs):
    xfail = kwargs.pop('xfail') if 'xfail' in kwargs.keys() else []
    arg_names = sorted(kwargs.keys())
    def _build_args_(in_tup = (), arg_index = 0):
        for val in kwargs[arg_names[arg_index]]:
            out_tup = (*in_tup, val)
            if arg_index < len(arg_names) - 1:
                yield from _build_args_(out_tup, arg_index + 1)
            else:
                if out_tup in xfail:
                    out_tup = pytest.param(*out_tup, marks = pytest.mark.xfail)
                yield out_tup
    return ','.join(arg_names), _build_args_()

@pytest.mark.parametrize(*__get_param_general_generator__(
    x = [0, 1],
    y = [2, 3],
    xfail = [(0, 2)]
    ))
def test_foo_xy(mock_fixture, x, y):
    mock_fixture.do_something((x, y))

@pytest.mark.parametrize(*__get_param_general_generator__(
    x = [0, 1],
    y = [2, 3],
    z = [0, 1, 2],
    xfail = [(0, 2, 1)]
    ))
def test_bar_xyz(x, y, z):
    assert not (x == 0 and y == 2 and z == 1)

(Thanks to yield from, this is Python 3.3 and above only.)

Upvotes: 5

Related Questions