CODEWITHSUNDEEP
testingrobotframework
Calyth
Calyth

Reputation: 1673

Is there a way to get Robot Framework to run test suites in a certain order?

Suppose I have 2 test suites in the local directory, foo and bar, and I want to run the test suite in the order of foo then bar.

I tried to run pybot -s foo -s bar ., but then it just goes and run bar then foo (i.e. in alphabetical order).

Is there a way to get pybot to run robot framework suites to be execute in the order that I define?

Upvotes: 15

Views: 42764

Answers (5)

ninjaniko
ninjaniko

Reputation: 1

I had success using a listener:

Listener.py:

class Listener(object):
    ROBOT_LISTENER_API_VERSION = 3

    def __init__(self):
        self.priorities = ['foo', 'bar']

    def start_suite(self, data, suite):
        #data.suites is a list of <TestSuite> instances
        data.suites = self.rearrange(data.suites)

    def rearrange(self, suites=[]):
        #Do some sorting of suites based on self.priorities e.g. using bubblesort
        n = len(suites)
        if n > 1:
            for i in range(0, n):
                for j in range(0, n-i-1):
                    #Initialize the compared suites with lowest priority
                    priorityA = 0
                    priorityB = 0
                    #If suite[j] is prioritized, get the priority of it
                    if str(suites[j]) in self.priorities:
                        priorityA = len(self.priorities)-self.priorities.index(str(suites[j]))
                    #If suite[j+1] is prioritized, get the priority of it
                    if str(suites[j+1]) in self.priorities:
                        priorityB = len(self.priorities)-self.priorities.index(str(suites[j+1]))
                    #Compare and swap if suite[j] is of lower priority than suite[j+1]
                    if priorityA < priorityB:
                        suites[j], suites[j+1] = suites[j+1], suites[j]             
        return arr

Assuming foo.robot and bar.robot are contained in a toplevel suite called 'tests', you can run it like this:

pybot --listener Listener.py tests/

This will rearrange childsuites on the fly. It's possible you can modify it upfront using a prerunmodifier instead.

Upvotes: 0

sjudǝʊ
sjudǝʊ

Reputation: 1565

Robot framework can use argument files that can be used to specify order of execution (docs):

This is from older docs (not online anymore):

Another important usage for argument files is specifying input files or directories in certain order. This can be very useful if the alphabetical default execution order is not suitable:

Basically, you create something similar to start up script.

--name My Example Tests
tests/some_tests.html
tests/second.html
tests/more/tests.html
tests/more/another.html
tests/even_more_tests.html

There is neat feature that from argument file you can call another argument file that can override previously set parameters. Execution is recursive, so you can nest as many argument files as you need

Another option would be to use start up script. Than you have to deal with other aspects like which operating system you are running test on. You could also use python for starting up script on multiple platforms. There is more in this section of docs

Upvotes: 17

binithb
binithb

Reputation: 2000

If there are multiple test case files in an RF directory , the execution order can be specified by giving numbers as prefixes to test case names , like this.

01__my_suite.html -> My Suite 02__another_suite.html -> Another Suite

Such prefixes are not included in the generated test suite name if they are separated from the base name of the suite with two underscores:

More details are here.

http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#execution-order

Upvotes: 12

OGrandeDiEnne
OGrandeDiEnne

Reputation: 888

You can use tagging.

Tag the tests as foo and bar so you can run each test separately:

pybot -i foo tests

or

pybot -i bar tests

and decide the order

pybot -i bar tests || pybot -i foo tests

or in a script.

The drawback is that you have to run the setup for each test.

Upvotes: 4

Racktash
Racktash

Reputation: 96

Would something like this be of any use?

pybot tests/test1.txt tests/test2.txt

So, to reverse:

pybot tests/test2.txt tests/test1.txt

Upvotes: 1

Related Questions