Reputation: 5873
I am seeing unexplained test failures when using Visual Studio 2022's VSTest's Logger::WriteMessage
from multiple threads. In normal runs, I see
Test Run Successful.
Total tests: 26
Passed: 26
Total time: 3.1651 Seconds
while occasionally, I instead see
Source array was not long enough. Check srcIndex and length, and the array's lower bounds.
at the end of a test, and something like
Test Run Failed.
Total tests: 17
Passed: 17
Total time: 2.2211 Seconds
at the end of such a test run.
In the IDE, this is noticeable by "Run Until Failure" running for 1000 iterations, but reporting passes for less than 1000 iterations (but no failures!).
This is the code that reproduces the problem. It is highly optimized to reproduce the problem after <10 test runs:
#include <thread>
#include <vector>
#include "CppUnitTest.h"
using namespace std::chrono_literals;
using Microsoft::VisualStudio::CppUnitTestFramework::Logger;
namespace NamespaceFoo {
TEST_CLASS (ClassFoo) {
#define TEST(NAME) \
TEST_METHOD (NAME) { \
std::vector<std::jthread> threads; \
for (int i = 0; i < 10; i++) \
threads.emplace_back([] { \
std::this_thread::sleep_for(100ms); \
Logger::WriteMessage("Message"); \
}); \
}
TEST(A)
TEST(B)
TEST(C)
TEST(D)
TEST(E)
TEST(F)
TEST(G)
TEST(H)
TEST(I)
TEST(J)
TEST(K)
TEST(L)
TEST(M)
TEST(N)
TEST(O)
TEST(P)
TEST(Q)
TEST(R)
TEST(S)
TEST(T)
TEST(U)
TEST(V)
TEST(W)
TEST(X)
TEST(Y)
TEST(Z)
};
} // namespace NamespaceFoo
This is in Visual Studio 2022 v17.10.4 using MSVC 14.34.31931 (from Visual Studio 2022 v17.4).
To run it:
"%VS2022INSTALLDIR%\Common7\Tools\VsDevCmd.bat" -arch=amd64
for /L %i in () do @(vstest.console.exe Demo_VSTest.dll)
Logger::WriteMessage
seems to be hardly documented, so I wonder if my use of it from multiple threads may cause the problems I am seeing. I have noticed that locking a mutex
before writing log messages helps, but I wonder if that is necessary.
Upvotes: 0
Views: 74