CODEWITHSUNDEEP
pythonubuntupycharmpytest
user3840896
user3840896

Reputation: 111

Py.test collection phase taking very long

I am really quite new to development in Python in general, let alone testing with pytest. My problem is that the pytest collection phase runs unusually slow. I am specifying the test directory which contains only a handful of files with only one file containing three tests. The collection takes pretty much a whole minute, after which the actual tests run in under a few seconds. I have looked at similar questions but couldn't find a solution. I don't think it matters (as py.test is slow even from the command line) but I am using the pycharm IDE. The OS is Ubuntu.

This may be relevant: If I terminate the process after a few seconds I usually end up with a stacktrace ending as follows:

<A FEW LINES OMITTED...>
File "/usr/local/lib/python2.7/dist-packages/_pytest/core.py", line 413, in __call__
    return self._docall(methods, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/core.py", line 424, in _docall
    res = mc.execute()
  File "/usr/local/lib/python2.7/dist-packages/_pytest/core.py", line 315, in execute
    res = method(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/helpconfig.py", line 27, in pytest_cmdline_parse
    config = __multicall__.execute()
  File "/usr/local/lib/python2.7/dist-packages/_pytest/core.py", line 315, in execute
    res = method(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 636, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 747, in parse
    self._preparse(args)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 709, in _preparse
    self._initini(args)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 704, in _initini
    self.inicfg = getcfg(args, ["pytest.ini", "tox.ini", "setup.cfg"])
  File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 861, in getcfg
    if exists(p):
  File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 848, in exists
    return path.check()
  File "/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line 352, in check
    return exists(self.strpath)
  File "/usr/lib/python2.7/genericpath.py", line 18, in exists
    os.stat(path)
KeyboardInterrupt

Or sometimes...

<STACK TRACE...>
  File "/usr/local/lib/python2.7/dist-packages/py/_iniconfig.py", line 50, in __init__
    f = open(self.path)
KeyboardInterrupt

Maybe one of the two last calls before the KeyboardInterrupt is very slow?

Please do ask for more detail should you require it!

Cheers!

Upvotes: 11

Views: 10606

Answers (4)

Ben Sturmfels
Ben Sturmfels

Reputation: 1483

Having a large number of files in a sub-directory can cause slow test collection, even non-Python files like images and PDFs. See this similar question.

Upvotes: 1

John Targaryen
John Targaryen

Reputation: 1192

My solution was based off of this answer, where I did pytest dir/to/tests and it skipped the collection step entirely.

Upvotes: 1

xjcl
xjcl

Reputation: 15309

Add PYTHONDONTWRITEBYTECODE=1 to your environment variables!

  • Windows Batch: set PYTHONDONTWRITEBYTECODE=1
  • Unix: export PYTHONDONTWRITEBYTECODE=1
  • subprocess.run: Add keyword env={'PYTHONDONTWRITEBYTECODE': '1'}

Note that the first two options are only valid for your current terminal session.

Here is how I found this out: pytest was being unusably slow from the command line, but working fine from within PyCharm. Copying the PyCharm command into cmd.exe (executes a small helper script) also was unusuably slow. Thus I printed out the environ variables at os.environ and tried it with that -- and it was fast! Then I eliminated each one-by-one.

Upvotes: 2

smoquet
smoquet

Reputation: 371

I had the same problem. My fix was to set the Working directory setting in the Run/Debug Configuration to the folder where manage.py is located.

Upvotes: 1

Related Questions