Script raising Exception as per python's logic but Test Case still passed in robot framework

Question

I am a new bee to Robot and having issue with my script execution part .

My case is failing correctly as per python's logic . However robot is still passing it . I am quite certain that i have missed out some thing on robot script end like some keyword maybe.

Any inputs and suggestion are more than welcome

I tried Run keywork if but i do not know how to use it . Please see both my python and robot code below

robot script:

*** Settings ***
Documentation       TC2
Library             exception
#Library           common_functions

*** Keywords ***

Run Keyword if 'FAIL' in $output

*** Test Cases ***
TEST CASE TWO
    validation

python code:

import os
import re


def validation():
    try:
        if 'loopback is up, line protocol is up' in open('1.txt').read():
            print("string found")

        else:
            print ("string not found")
            raise Exception("string not found")

    except:
        cleanup()

def cleanup():
    print("cleanup is in progress")


validation()

Expected result :

1. Python script showing : string not found cleanup is in progress

2. Robot should show fail

Actual output :

1. Python script showing : string not found cleanup is in progress

2. Robot script showing PASS

Answer 1

Do not catch the exception in your Python code, let that fail the robot execution. You can do the cleanup in the test teardown section.

import os
import re


def validation():
    if 'loopback is up, line protocol is up' in open('1.txt').read():
        print("string found")

    else:
        print ("string not found")
        raise RuntimeError("string not found")


def cleanup():
    print("cleanup is in progress")

---

*** Settings ***
Documentation       TC2
Library             exception

*** Test Cases ***
TEST CASE TWO
    validation
    [Teardown]    cleanup

Answer 2

(First guess, knowing nothing about the robot framework, but about to dive in): Your robot script is looking for FAIL in $output. Your output contains string not found cleanup is in progress (perhaps with different formatting, as you've printed twice without specifying to not print newlines). There is no FAIL in your output. Perhaps you meant to have

Run Keyword if 'string not found' in $output

in your robot script, or

print("FAIL: string not found")

in your validation code?

UPDATE: Now that I have spent some time with the documentation (they don't make it easy to learn to do something simple), I can say that Bence Kaulics answer is mostly correct, perhaps depending on what version you're running. For me, the test as shown in his answer fails with an IndentationError due to the try without except. Here's what I have that works (as in, fails for the expected reason, mostly):

.robot file:

*** Settings ***
Documentation       TC2
Library             exception.py

*** Test Cases ***
TEST CASE TWO
    validation

exception.py:

def validation():
    try:
        if 'loopback is up, line protocol is up' in open('1.txt').read():
            result = 'string found'
        else:
            result = 'string not found'
            raise RuntimeError(result)
    except Exception as e:
        result = 'string not found'
        raise RuntimeError(result)
    finally:
        print(result)
        cleanup()


def cleanup():
    print("cleanup is in progress")

With the above, I get the output:

(robottestframework) ~/s/l/robottestframework> robot -L TRACE keyword_driven.robot  
==============================================================================
Keyword Driven :: TC2                                                         
==============================================================================
TEST CASE TWO                                                         | FAIL |
string not found
------------------------------------------------------------------------------
Keyword Driven :: TC2                                                 | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================

Note that this changes the semantics of what you are testing: instead of checking the value of stdout, you are running a keyword imported from a library, and failing if there is an exception raised. IMO, that's a better way to go than checking stdout. However, if what you really need to do is check the value of stdout, you'll need something more like:

.robot file:

*** Settings ***
Documentation       TC2
Library             Process
Suite Teardown      Terminate All Processes     kill=True

*** Test Cases ***
TEST CASE TWO
    ${result} =     Run Process     python      exception.py
    should not contain      ${result.stdout}        string not found

exception.py file:

def validation():
    result = ''
    try:
        if 'loopback is up, line protocol is up' in open('1.txt').read():
            result = 'string found'
        else:
            result = 'string not found'
            raise RuntimeError(result)
    except Exception as e:
        result = 'string not found'
    finally:
        print(result)
        cleanup()
    return result


def cleanup():
    print("cleanup is in progress")


validation()

With these changes, I get:

(robottestframework) ~/s/l/robottestframework> robot -L TRACE keyword_driven.robot  
==============================================================================
Keyword Driven :: TC2                                                         
==============================================================================
TEST CASE TWO                                                         | FAIL |
'string not found
cleanup is in progress' contains 'string not found'
------------------------------------------------------------------------------
Keyword Driven :: TC2                                                 | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================