CODEWITHSUNDEEP
pythonpytest
user8615607
user8615607

Reputation: 197

pytest -k expression failing when trying to match multiple parameterized tests

Currently trying to get a pytest -k expression to work which will run multiple parameterized tests.
For example, this runs fine:

$ pytest tools/test_pinger_tracer.py  --collect-only                                                                                                                        
=============================================== test session starts ========================================
platform linux -- Python 3.7.4, pytest-3.7.0, py-1.8.0, pluggy-0.13.0
rootdir: /home/as2863/pythonProjects/p1-automation, inifile: pytest.ini
plugins: csv-2.0.1, check-0.3.5, pylama-7.6.6, dependency-0.4.0, instafail-0.4.0, ordering-0.6, repeat-0.7.0, reportportal-5.0.3
collected 4 items
<Package '/home/as2863/pythonProjects/p1-automation/tools'>
  <Module 'test_pinger_tracer.py'>
    <Function 'test_pinger[ping-pinger_topoA_L0_L10.yml]'>
    <Function 'test_tracer[traceroute-pinger_topoA_L0_L10.yml]'>
    <Function 'test_tracer[traceroute-mplstracer_edgetopoA_L0_L10.yml]'>
    <Function 'test_tracer[mpls traceroute-mplstracer_edgetopoA_L0_L10.yml]'>

================================================ no tests ran in 0.01 seconds ==============================

And if I want to run just one test with the -k option, it works fine:

pytest tools/test_pinger_tracer.py  --collect-only -k "test_pinger[ping-pinger_topoA_L0_L10.yml]"
============================================== test session starts ===========================================
platform linux -- Python 3.7.4, pytest-3.7.0, py-1.8.0, pluggy-0.13.0
rootdir: /home/as2863/pythonProjects/p1-automation, inifile: pytest.ini
plugins: csv-2.0.1, check-0.3.5, pylama-7.6.6, dependency-0.4.0, instafail-0.4.0, ordering-0.6, repeat-0.7.0, reportportal-5.0.3
collected 4 items / 3 deselected
<Package '/home/as2863/pythonProjects/p1-automation/tools'>
  <Module 'test_pinger_tracer.py'>
    <Function 'test_pinger[ping-pinger_topoA_L0_L10.yml]'>

================================ 3 deselected in 0.01 seconds ==================

However, if I try to use an "or" statement in the -k expression to run two tests, pytest crashes...

$ pytest tools/test_pinger_tracer.py  --collect-only -k "test_pinger[ping-pinger_topoA_L0_L10.yml] or test_tracer[traceroute-mplstracer_edgetopoA_L0_L10.yml]"
================================ test session starts ======================================
platform linux -- Python 3.7.4, pytest-3.7.0, py-1.8.0, pluggy-0.13.0
rootdir: /home/as2863/pythonProjects/p1-automation, inifile: pytest.ini
plugins: csv-2.0.1, check-0.3.5, pylama-7.6.6, dependency-0.4.0, instafail-0.4.0, ordering-0.6, repeat-0.7.0, reportportal-5.0.3
collecting 4 items                                                                                                                                                                                                                                     <Package '/home/as2863/pythonProjects/p1-automation/tools'>
  <Module 'test_pinger_tracer.py'>
    <Function 'test_pinger[ping-pinger_topoA_L0_L10.yml]'>
    <Function 'test_tracer[traceroute-pinger_topoA_L0_L10.yml]'>
    <Function 'test_tracer[traceroute-mplstracer_edgetopoA_L0_L10.yml]'>
    <Function 'test_tracer[mpls traceroute-mplstracer_edgetopoA_L0_L10.yml]'>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/_pytest/main.py", line 214, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/manager.py", line 92, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/manager.py", line 86, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/_pytest/main.py", line 224, in pytest_collection
INTERNALERROR>     return session.perform_collect()
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/_pytest/main.py", line 431, in perform_collect
INTERNALERROR>     session=self, config=self.config, items=items
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/manager.py", line 92, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/manager.py", line 86, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/_pytest/mark/__init__.py", line 155, in pytest_collection_modifyitems
INTERNALERROR>     deselect_by_keyword(items, config)
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/_pytest/mark/__init__.py", line 124, in deselect_by_keyword
INTERNALERROR>     if keywordexpr and not matchkeyword(colitem, keywordexpr):
INTERNALERROR>   File "/home/as2863/python-venv/p1_netmiko_exper4/lib/python3.7/site-packages/_pytest/mark/legacy.py", line 95, in matchkeyword
INTERNALERROR>     return eval(keywordexpr, {}, mapping)
INTERNALERROR>   File "<string>", line 1, in <module>
INTERNALERROR> AttributeError: 'bool' object has no attribute 'yml'

====================================== no tests ran in 0.02 seconds==================================================

Note, if I take out the "square brackets and the "period" from the string expressions, then it won't crash but it won't match anything either:

$ pytest tools/test_pinger_tracer.py  --collect-only -k "test_pingerping-pinger_topoA_L0_L10yml or test_tracertraceroute-mplstracer_edgetopoA_L0_L10yml"
================================================================================================================= test session starts ==================================================================================================================
platform linux -- Python 3.7.4, pytest-3.7.0, py-1.8.0, pluggy-0.13.0
rootdir: /home/as2863/pythonProjects/p1-automation, inifile: pytest.ini
plugins: csv-2.0.1, check-0.3.5, pylama-7.6.6, dependency-0.4.0, instafail-0.4.0, ordering-0.6, repeat-0.7.0, reportportal-5.0.3
collected 4 items / 4 deselected

====================================== 4 deselected in 0.01 seconds ==================================================================

Any suggestions for being able to select multiple parameterized pytests to run? Thx Al

Upvotes: 2

Views: 1480

Answers (1)

hoefling
hoefling

Reputation: 66251

Turning my comment into an answer, since pytest==6.0 was released yesterday:

This was fixed in version 6.0 with an introduction of a new parser for -k argument. The old behaviour of evaling -k argument as an expression was removed. Anyone interested in the actual implementation should take a look at #7122's diff.

Upvotes: 1

Related Questions