CODEWITHSUNDEEP
bddspecflow
CamperWill
CamperWill

Reputation: 108

SpecFlow report all assertions even if one fails

Does anyone know if this is possible with SpecFlow? In the spirit of not having more than one assertion per test, I was hoping SpecFlow would treat each "Then" as a separate test.

I have a Scenario with multiple "Then" steps that looks something like this (snippet):

When a summary for day <day number> is requested
Then the summary well id should be "134134"
 And the summary well name should be "My Well --oops!"
 And the summary violated rules should be <violations>
 And the summary variance should be <variance>

Examples: 
| day number | violations      | variance |
| 0          | BadTotal,HpOnLp | -33      |
| 3          | BadTotal        | -133.33  |
| 5          |                 | 0        |

The second assertion "My Well --oops!" should fail. What I want is for SpecFlow to test the assertions that follow

I get:

When a summary for day 0 is requested
-> done: DprSummarySteps.WhenASummaryForDayIsRequested(0) (0.0s)
Then the summary well id should be "134134"
-> done: DprSummarySteps.ThenTheSummaryWellIdShouldBe("134134") (0.0s)
And the summary well name should be "My Well --oops!"
-> error: Assert.AreEqual failed. Expected:<My Well --oops!>. Actual:<My Well>. 
And the summary violated rules should be BadTotal,HpOnLp
-> skipped because of previous errors
And the summary variance should be -33
-> skipped because of previous errors
Assert.AreEqual failed. Expected:<My Well --oops!>. Actual:<My Well>.

What I want:

When a summary for day 0 is requested
-> done: DprSummarySteps.WhenASummaryForDayIsRequested(0) (0.0s)
Then the summary well id should be "134134"
-> done: DprSummarySteps.ThenTheSummaryWellIdShouldBe("134134") (0.0s)
And the summary well name should be "My Well --oops!"
-> error: Assert.AreEqual failed. Expected:<My Well --oops!>. Actual:<My Well>. 
And the summary violated rules should be BadTotal,HpOnLp
-> done: DprSummarySteps.ThenTheViolatedRulesShouldBe("BadTotal,HpOnLp") (0.0s)
And the summary variance should be -33
-> done: DprSummarySteps.ThenTheVarianceShouldBe(-33) (0.0s)
Assert.AreEqual failed. Expected:<My Well --oops!>. Actual:<My Well>.

Upvotes: 2

Views: 2680

Answers (3)

Prakhar Londhe
Prakhar Londhe

Reputation: 1471

I have identified one way to do this. So what happens internally in specflow, before running any step, the TestExecutionEngine checks the LastExecutionStatus of Scenario, and it does not proceed if it is not OK.. So what we can do is, in [AfterStep] hook, add the following line :

typeof(ScenarioContext).GetProperty("ScenarioExecutionStatus").SetValue(this.ScenarioContext, ScenarioExecutionStatus.OK);

Replace this.ScenarioContext with whatever object with type ScenarioContext exist. This will make the current status as OK and will allow you to proceed to next step.

Keep in mind this won't allow you to catch all assert failures in a single step.

Upvotes: 0

Shannon McRae
Shannon McRae

Reputation: 245

You can do this now if you upgrade to Specflow 2.4 and are using Nunit, see something like https://github.com/nunit/docs/wiki/Multiple-Asserts

Assert.Multiple(() =>
{
    Assert.AreEqual(5.2, result.RealPart, "Real part");
    Assert.AreEqual(3.9, result.ImaginaryPart, "Imaginary part");
});

Upvotes: 1

Sam Holder
Sam Holder

Reputation: 32954

I don't believe that you can do this but my question to you would be why do you want that? With any unit test it will fail at the first assertion. you wouldn't expect a test to continue execution after an assertion failed and this is no different. Surely knowing that the test failed for some reason is enough. In this specific case you might be able to make separate independent assertions which provide useful info, but in the general case assertions that come after one that has failed might be completely meaningless.

If you want to have each assertion independent of the others then you need to break it into several scenarios with each of you current And steps as its own Then step. You might be able to use a Background step to do the common setup.

I'm not sure that this will help you though as it seems like your assertions are all related to the examples so you end up needing to repeat the examples.

Upvotes: 1

Related Questions