CODEWITHSUNDEEP
c#timeoutmstestspecflowranorex
Benny Meade
Benny Meade

Reputation: 612

Specflow ignores Visual Studio Unit Test Attribute 'Timeout'

I have an automation suite using: Mstest, Specflow and Ranorex

The WPF application under test is large and interacts with many applications like MS Word, Browsers, Sharepoint, Abobe etc. So from time to time the flow of the automation testing has a hiccup and the test hangs. This hanging time then continues until the VS/MsTest 30 minutes default timeout expires, before the application is killed and the next test can begin.

I want to decrease this wait time. There are 2 options that come up constantly in google:

  1. set [TestMethod, Timeout(x)]
  2. edit the Local.testsettings Test Timeout value

But neither of these seem to work under these conditions because:

  1. Specflow ignores any MsTest attribute added as seen below:

        [TestMethod, Timeout(1000)]
    [When(@"I test")]
    public void WhenITest()
    {
        // some code
    }

  2. "If you use a .testsettings file... does not allow you to run tests from third-party test frameworks." Source

Therefore I need a workaround! And it seems the best way to do this is directly with Specflow hooks (as suggested here in Answer 3). But to implement this into a huge framework and ensure every feature file has the tag added seems potential for human error.

Any anyone overcome a similar issue?

Upvotes: 2

Views: 3525

Answers (3)

Benny Meade
Benny Meade

Reputation: 612

So it turns out that the option 2 above eventually worked for me: * edit the Local.testsettings Test Timeout value

The fix was that I needed to specify the Local.testsettings as an additional parameter in the TeamCity Build Steps.

Example: TeamCity - Build Step

  • Runner Type = MsTest

  • List assembly files = /MyProject/UITests.dll

  • Additional command line parameters = /testsettings:"Trunk/Tests/AutomationTests/ProjectName/local.testsettings"

Upvotes: 0

Keysharpener
Keysharpener

Reputation: 504

Out of curiosity, are you debugging the problematic test? As far as I know, the timeout is not taken into account by MsTest during a debugging session.

If not, you could add a timer to your test initialization, set to the timeout value you wish. To add to Sam Holder's answer, you could have (with MsTest syntax):

    [TestInitialize]
    public void Initialize()
    {
        Timer timer = new Timer(FailTheTest, null, 0, timeoutValue);            
    }

    private void FailTheTest(object state)
    {
        Assert.Fail("Insert here your timeout message");
    }

Be sure to correctly clean up your test environment afterwards, so as not to leave documents, or service calls hanging.

Upvotes: 1

Sam Holder
Sam Holder

Reputation: 32936

You don't need to add it to every feature file if you are happy to add it to all tests by default. Just add a [BeforeScenario] with no tag and insert your additional code in there and it will be added to every test.

You can also 'opt-out' some tests by checking if the opt out tag is present and not executing the code in that case:

[BeforeScenario()]
public static void BeforeWebScenario()
{
    if(!ScenarioContext.Current.ScenarioInfo.Tags.Contains("noReducedTimeout"))
        AddReducedTimeout;
}

I'm not sure how you will add the MSTest timeout attribute to your test, but as you seem to think the other answer provides that information then I'll leave that as an exercise for the student.

If this doesn't work then you should be able to do this by adding a plugin for specflow which will allow the unit test generator to augment the generated tests with this timeout. Unfortunately there is not a great deal of detail on how to do this out there, but this blog post should point you in the right direction.

If you do write a plugin, writing one that adds any attributes on the parent step definition could be generically useful :)

Upvotes: 1

Related Questions