CODEWITHSUNDEEP
pythonrobotframework
Santosh
Santosh

Reputation: 1

How to run test cases as part of suite setup?

I am using robotframework to run bunch of python functions as test cases. I tagged bunch of test cases as setup and bunch of test cases as cleanup. I use -i option in robot to run various tests cases. Typically I run -i setup for setting up and -i cleanup to clean up the test suite. However, if a test case in setup fails , all the test cases are still executed. I am looking for an ability to mark bunch of test cases as suite setup and suite teardown, and run suite teardown if suite setup fails automatically without running any test cases in between. I tried the Suite Setup with keywords option, but with that I am moving my test cases to keywords section and I am loosing the tags functionality.

Original robot file

*** Settings ***

Suite Setup
Suite Teardown

*** Test Cases ***

setup_1
    [Tags]    setup    smoke    1
    ${result}   ${result_str} =     test.test.TestSetup.test_setup_1
    Should Be Equal  ${result}  ${TRUE}
setup_2
    [Tags]    setup    smoke    1
    ${result}   ${result_str} =     test.test.TestSetup.test_setup_2
    Should Be Equal  ${result}  ${TRUE}
setup_3
    [Tags]    setup    smoke    1
    ${result}   ${result_str} =     test.test.TestSetup.test_setup_3
    Should Be Equal  ${result}  ${TRUE}

TestCase 1
    [Tags]      tc
    ${result}   ${result_str} =     test.test.my_func.test_func_1
    Should Be Equal  ${result}  ${TRUE}
TestCase 2
    [Tags]      tc
    ${result}   ${result_str} =     test.test.my_func.test_func_2
    Should Be Equal  ${result}  ${TRUE}

cleanup_1
    [Tags]    cleanup    smoke    1
    ${result}   ${result_str} =     test.test.TestCleanup.test_cleanup_1
    Should Be Equal  ${result}  ${TRUE}
cleanup_2
    [Tags]    cleanup    smoke    1
    ${result}   ${result_str} =     test.test.TestCleanup.test_cleanup_2
    Should Be Equal  ${result}  ${TRUE}

Robot file with keywords

*** Settings ***
Suite Setup     Run keywords
...     setup_1
...     AND setup_2
...     AND setup_3

Suite Teardown  Run keywords
...     cleanup_1
...     AND cleanup_2

*** Keywords ***
setup_1
    [Tags]    setup    smoke    1
    ${result}   ${result_str} =     test.test.TestSetup.test_setup_1
    Should Be Equal  ${result}  ${TRUE}
setup_2
    [Tags]    setup    smoke    1
    ${result}   ${result_str} =     test.test.TestSetup.test_setup_2
    Should Be Equal  ${result}  ${TRUE}
setup_3
    [Tags]    setup    smoke    1
    ${result}   ${result_str} =     test.test.TestSetup.test_setup_3
    Should Be Equal  ${result}  ${TRUE}

cleanup_1
    [Tags]    cleanup    smoke    1
    ${result}   ${result_str} =  test.test.TestCleanup.test_cleanup_1
    Should Be Equal  ${result}  ${TRUE}
cleanup_2
    [Tags]    cleanup    smoke    1
    ${result}   ${result_str} =     test.test.TestCleanup.test_cleanup_2
    Should Be Equal  ${result}  ${TRUE}

*** Test Cases ***
TestCase 1
    [Tags]      tc
    ${result}   ${result_str} =     test.test.my_func.test_func_1
    Should Be Equal  ${result}  ${TRUE}
TestCase 2
    [Tags]      tc
    ${result}   ${result_str} =     test.test.my_func.test_func_2
    Should Be Equal  ${result}  ${TRUE}

Is there any way to mark tests that are tagged with setup as Suite Setup and mark tests that are tagged with cleanup as Suite Teardown and still have an ability to run individual tests in setup and teardown?

Upvotes: 0

Views: 1201

Answers (2)

Sameem
Sameem

Reputation: 849

Suite Setup and Suite Teardown are meant to execute a series of steps before and after the execution of the test cases. You can think of Suite Setup similar to Pre-requisite/Pre-condition in manual test cases. It only makes sense for the entire suite to not be executed when the setup fails(as is the case with robot).

If the keywords in suite setup and tear-down are themselves test cases, I would suggest you to execute them as a test case in another suite.

There are a couple of workarounds for your case, provided the following assumptions are true:

Assumptions:

  1. The individual keywords in suite setup and tear-down are independent of each other (i.e), if there are 2 keywords in suite setup namely Keyword 1 and Keyword 2, they are not dependent on each other. Similarly for keywords in suite tear-down.
  2. The test cases are dependent on only one of the setup and tear-down keywords (i.e), if there are 2 keywords in suite setup namely Keyword 1 and Keyword 2, the each test case is dependent on only one of the keywords.

Workaround 1:

Use the Test Setup and Test Teardown settings. This allows you to fail test cases only for which the setup fails which can then be re-run on failure.

Workaround 2:

Move the test cases to 2 different suites and use the Suite Setup and Suite Teardown to configure the respective nodes.

Note: In both the above cases, you would have to move the test cases in the setup and tear-down process to a different suite. And yes, you would end up with more than one suite to maintain, but I believe it would make it much easier to maintain such a script instead of making a mess of a single suite.

If the mentioned assumptions are not valid, it really doesn't make sense to re-run a failed test in your suite.

Upvotes: 0

Bryan Oakley
Bryan Oakley

Reputation: 385830

How to run test cases as part of suite setup?

You can't. Test cases aren't something you can explicitly call from something else.

If you're creating a reusable component that needs to run in a suite setup, it needs to be a user keyword rather than a test case.

If you want an entire suite to stop running when a single test case fails, you can use the -X/--exitonfailure command line option. Another option would be to create a test teardown that calls the built-in keyword Fatal error

Upvotes: 1

Related Questions