How do I start an automated test (Codedui) externally?

Question

CodedUI tests use the UnitTest Adapter of the QualityTools namespace as shown in this stack trace upon entry of a test using TestExplorer (debug test selection).

Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter.DefaultTestMethodInvoke(object[] args)   Unknown
Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.DefaultTestMethodDecorator.Invoke(object[] args)  Unknown
Microsoft.VisualStudio.QualityTools.CodedUITestFramework.dll!Microsoft.VisualStudio.TestTools.UITesting.CodedUITestInvoker.Invoke(object[] parameters)  Unknown
Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter.RunTestMethod()  Unknown
Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter.ExecuteTest()    Unknown
Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter.Execute(Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestResult result)   Unknown
Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestRunner.ExecuteSingleTest(Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter executer, Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestResult result, Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement test, Microsoft.VisualStudio.TestTools.Execution.ITestContext testContext, Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapterContext userContext, bool isLoadTest)    Unknown
Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestRunner.ExecuteSingleTest(Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter executer, Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement test, Microsoft.VisualStudio.TestTools.Execution.ITestContext testContext, Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapterContext userContext, bool isLoadTest)   Unknown
Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestRunner.Run(Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement test, Microsoft.VisualStudio.TestTools.Execution.ITestContext testContext, bool isLoadTest, bool useMultipleCpus)  Unknown
[AppDomain (QTAgent32.exe, #1) -> AppDomain (TestAppDomain: adad0b78-8b62-491f-8c88-b41c3e647a1f, #2)]  
Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter.Run(Microsoft.VisualStudio.TestTools.Common.ITestElement testElement, Microsoft.VisualStudio.TestTools.Execution.ITestContext testContext, bool isLoadTest)   Unknown
Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter.Run(Microsoft.VisualStudio.TestTools.Common.ITestElement testElement, Microsoft.VisualStudio.TestTools.Execution.ITestContext testContext)    Unknown
Microsoft.VisualStudio.QualityTools.AgentObject.dll!Microsoft.VisualStudio.TestTools.Agent.AgentExecution.CallAdapterRunMethod(object obj)  Unknown

The TestContext properties have some exposed properties of which we can change some behaviors.

Each CODEDUI class must use this attribute in order for Test Runner, and Load Tests et.al. to "Find" the tests within an assembly.

This appears to be how the "CodedUI" framework operates minus one small detail. Upon entry to a Codedui test, a set of steps are executed synchronously. Upon the last step being executed, that instance of the "run" is terminated automatically. Tests pass automatically if no assertions are made. etc. etc.

But what about taking control over when the test ends? Does anyone know of a way to hook up an asynchronous task (implied delayed callback) and holding up the test completion until the continuation has happened.

I am guessing and tests have shown that the current support of CodeUI is that everything must be synchronous.

Here's my thoughts on this, I could from the test runner start the Async test. The test runner completes and this test passed indicating Async test started. Upon completion I could restart an automated test.

So the question boils down to this: How can I start an automated test externally? This will lead to the working design as I know that any async test is marked complete when test runner is done. It even kills all async work... which means those threads must be under the test runner process.

Answer 1

@echo off set max=.set count=. echo starting test execution echo ======================= "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\mstest.exe" /testcontainer:E:\Practice\TestingSample\TestingSample\bin\Debug\TestingSample.dll /test:fn_TestMethod1 /test:fn_TestMethod2 /resultsfile:E:\SachintestResAut3.trx"

"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\mstest.exe" /testcontainer:E:\Practice\TestingSample\TestingSample\bin\Debug\TestingSample.dll /test:fn_TestMethod2 /resultsfile:E:\SachintestResAut3.trx"

echo all done @exit

Answer 2

Reference the assembly and call the test method.