How can I run only tests with specific marker attribute in Pytest?

Question

I'm using @pytest.mark to uniquely identify specific tests, therefore I created my custom marker.

@pytest.mark.key

I'm using it as such:

@pytest.mark.key("test-001")
def test_simple(self):
    self.passing_step()
    self.passing_step()
    self.passing_step()
    self.passing_step()
    assert True

Now from the console I would like to run all tests with the marked key "test-001". How can I achieve this?

What i'm looking for is something like this:

pypi.org/project/pytest-jira/0.3.6

where a test can be mapped to a Jira key. I looked at the source code for the link but i'm unsure how to achieve it in order for me to run specific tests. Say I only wanna run the test with the key "test-001".

Answer 1

Pytest does not provide this out of the box. You can filter by marker names using the -m option, but not by marker attributes.
You can add your own option to filter by keys, however. Here is an example:

conftest.py

def pytest_configure(config):
    # register your new marker to avoid warnings
    config.addinivalue_line(
        "markers",
        "key: specify a test key"
    )


def pytest_addoption(parser):
    # add your new filter option (you can name it whatever you want)
    parser.addoption('--key', action='store')


def pytest_collection_modifyitems(config, items):
    # check if you got an option like --key=test-001
    filter = config.getoption("--key")
    if filter:
        new_items = []
        for item in items:
            mark = item.get_closest_marker("key")
            if mark and mark.args and mark.args[0] == filter:
                # collect all items that have a key marker with that value
                new_items.append(item)
        items[:] = new_items

Now you run something like

pytest --key=test-001

to only run the tests with that marker attribute.

Note that this will still show the overall number of tests as collected, but run only the filtered ones. Here is an example:

test_key.py

import pytest

@pytest.mark.key("test-001")
def test_simple1():
    pass

@pytest.mark.key("test-002")
def test_simple2():
    pass


@pytest.mark.key("test-001")
def test_simple3():
    pass

def test_simple4():
    pass

$ python -m pytest -v --key=test-001 test_key.py

...
collected 4 items

test_key.py::test_simple1 PASSED
test_key.py::test_simple3 PASSED
================================================== 2 passed in 0.26s ==================================================

---

UPDATE for pytest >= 8.3

As mentioned by MAndrás in the comments, support for keyword arguments in markers has been added in pytest 8.3.
It will not work for positional arguments as used in the example above, but if rewriting the markers to use keyword arguments, e.g.:

import pytest

@pytest.mark.key(name="test-001")
def test_simple1():
    pass

@pytest.mark.key(name="test-002")
def test_simple2():
    pass

...

pytest now supports filtering by these arguments out of the box:

$ python -m pytest -v -m key(name=\"test-001\") test_key.py
...
collected 4 items

test_key.py::test_simple1 PASSED
test_key.py::test_simple3 PASSED
================================================== 2 passed in 0.28s ==================================================

Note that the apostrophes have to be escaped for the command line to be correctly passed.

Answer 2

You can create custom markers. *You can see Working with custom markers.

First, register orange, apple and pineapple markers to [pytest] in pytest.ini as shown below. *You can still use custom markers without registration but there are warnings so registration is needed:

# "pytest.ini"

[pytest]
markers =
    orange: Orange marker
    apple: Apple marker
    pineapple: Pineapple marker

Then, use them as shown below:

import pytest

@pytest.mark.orange
def test1():
    assert True

@pytest.mark.apple
def test2():
    assert True

@pytest.mark.pineapple
def test3():
    assert True

Then, if you run pytest with -k apple as shown below:

pytest -k apple

Then, apple and pineapple are run as shown below. *-k can run the markers which contain the given string:

import pytest

@pytest.mark.orange # Not run
def test1():
    assert True

@pytest.mark.apple # Run
def test2():
    assert True

@pytest.mark.pineapple # Run
def test3():
    assert True

And, if you run pytest with -m apple as shown below:

pytest -m apple

Then, only apple is run as shown below. *-m can run the markers which is exactly same as the given string:

import pytest

@pytest.mark.orange # Not run
def test1():
    assert True

@pytest.mark.apple # Run
def test2():
    assert True

@pytest.mark.pineapple # Not run
def test3():
    assert True

Lastly, you can check the markers with --markers as shown below:

$ pytest --markers
@pytest.mark.orange: Orange marker

@pytest.mark.apple: Apple marker

@pytest.mark.pineapple: Pineapple marker
...

Answer 3

you can run pytest with -m option c

check below command:

pytest -m 'test-001' <your test file>