Is running unittest tests for Python under Visual Studio always this slow?

Question

I have a small Python project, just a few files and a few test files. There are 32 tests. I have been using Visual Studio 2015 on Windows 10 to write the code, and it works great for running the code, editing, and so on. It even found all the unit tests, and when I run them, it'll put green and red marks next to them depending on whether they pass or fail.

However, when I choose "Run All" in Visual Studio (that is, run the tests without debugging), it takes 19 seconds to run the 32 tests, and they're all taking about 300-400ms each to run. At first, I just thought, well, I guess running unit tests is just slower in Python.

But if I drop out to a command line, and do:

Python -m unittest test1.py test2.py test3.py [etc]

Then it runs the 32 tests successfully in 13ms. So, it's over 1000 times faster.

I can't seem to find any options or settings anywhere in Visual Studio that would allow me to change anything about testing. I can understand it taking a little longer because it records each test individually and marks the pass/fail state next to the item in the test explorer. But 1000 times slower? Nope.

At the moment, 19s isn't too long to wait. But when this hits 100 or 1000 tests, I'm going to be looking at large numbers of minutes to run the unit tests. Or just dropping out to the command line and not having nicely collected information about any tests that might have failed.

Has anyone else had this problem, or can anyone imagine what might be causing it? Or am I just stuck with it?

Answer 1

The short answer is that this is because it runs every test separately (i.e. it's as if you did python -m unittest test1.py; python -m unittest test2.py; ...).