CODEWITHSUNDEEP
.netvisual-studiopinvokemstest
Richard Beier
Richard Beier

Reputation: 1732

Can I force MSTest to use a new process for each test run?

We're using the VS 2010 test runner (MSTest) for automated functional testing. When we run tests from Visual Studio, VS creates a process called QTAgent32.exe, and it runs the tests in that process.

We've found that when we do multiple test runs, MSTest will reuse the same QTAgent32 process - the process ID does not change. This is a problem for us, since the code we're testing is P/Invoking to an unmanaged DLL. The DLL needs to be initialized only once during the lifetime of the process. We have an [AssemblyInitialize] method, but that executes once per test run. If we perform multiple test runs, it will execute more than once in the same process.

Every time we do a test run, MSTest creates a new appdomain; but these appdomains are all in the same process.

So I'm wondering: is there a way to tell the Visual Studio test runner to use a new process every time we run tests? I looked at the ".testsettings" configuration but didn't see anything relevant.

Upvotes: 13

Views: 4629

Answers (4)

Red Riding Hood
Red Riding Hood

Reputation: 2474

I created an (experimental) solution for MSTest + .Net Core in 2023

It requires https://www.nuget.org/packages/Tmds.ExecFunction

public class OutOfProcessTestAttribute : TestMethodAttribute
{
    private static readonly FunctionExecutor Executor = new FunctionExecutor(
    o =>
    {
        o.StartInfo.RedirectStandardError = true;
        o.OnExit = p =>
        {
            if (p.ExitCode != 0)
            {
                throw new Exception(p.StandardError.ReadToEnd());
            }
        };
    });

    public override TestResult[] Execute(ITestMethod testMethod)
    {
        var hostFilename = Process.GetCurrentProcess().MainModule.FileName;

        if (hostFilename.EndsWith("/testhost") || hostFilename.EndsWith("\\testhost.exe"))
        {
            return base.Execute(testMethod);
        }

        try
        {
            var obj = Activator.CreateInstance(testMethod.MethodInfo.DeclaringType!);
            Action action = (Action)testMethod.MethodInfo.CreateDelegate(typeof(Action), obj);
            Executor.Run(action);
            return new TestResult[1] { new() { Outcome = UnitTestOutcome.Passed } };
        }
        catch (Exception e)
        {
            return new TestResult[1] { new() { TestFailureException = e, Outcome = UnitTestOutcome.Failed } };
        }
    }
}

Now you can replace [TestMethod] with [OutOfProcessTest]

Important things to note:

  1. It will create a new instance of the Test Class, because I couldn't get reference to the original instance. Therefor try to make the method self contained as possible, the [TestInitialize] etc attributes won't apply.

  2. When running inside Visual Studio Test Host, Tmds.ExecFunction does not work because it cannot find the parent process (works fine from dotnet test though). I choose to just run the test normally, but you should modify the attribute to behave as you want (e.g. perhaps skip the test instead of run it)

  3. I did not test async functions, perhaps it will require some modification to support.

Upvotes: 0

tgriffin
tgriffin

Reputation: 535

VS 2013 and forward now has a setting for this under Test > Test Settings > Keep Test Execution Engine Running. Unchecking this selection will start up a new engine each run.

Upvotes: 0

Richard Beier
Richard Beier

Reputation: 1732

I was able to get this working after reading Wiktor's comment about FreeLibrary().

I used this class created by Mike Stall, which provides wrappers around LoadLibrary, GetProcAddress, and FreeLibrary. That way, I can load the library once in each test run, call the necessary methods, and then free the library at the end of the test run.

Mike Stall's code uses Marshal.GetDelegateForFunctionPointer, which converts an unmanaged function pointer to a managed delegate type.

I had to replace the [DllImport] extern declarations with declarations for delegate types. So I converted this:

[DllImport("asesignal.dll")]
public static extern bool ASESDK_Initialize(string licenseCode);

to this:

public delegate bool ASESDK_Initialize(string licenseCode);

Mike Stall's code contained examples with generic delegates (Action<T> etc.). But I couldn't get that working, so I created my own delegate types.

I can load the DLL dynamically like this:

_ht = new UnmanagedLibrary(@"c:\windows\system32\asesignal.dll");

To call a function, I do this:

var function = _ht.GetUnmanagedFunction<ASESDK_Initialize>("ASESDK_Initialize");
function(licenseCode);

Thanks Wiktor and np-hard for your help!

Upvotes: 1

np-hard
np-hard

Reputation: 5815

dont know how far you want to go with it, but one solution could be to create your unit test host

http://technet.microsoft.com/fr-fr/query/bb166558

this link shows how to create adapters, also you could then launch a new process for evertest, create a piped communication and tear it down after the test.

I know MS itself uses a different host for running tests under moles

http://research.microsoft.com/en-us/projects/pex/molestutorial.pdf

Upvotes: 6

Related Questions