Karate - how to configure the afterScenario hook in a called feature?

Question

I have the following use case. The top level feature file (test1.feature) calls two other feature files (test2.feature, test3.feature) like so:

test1.feature:

Feature:

Background:
    * print 'this is a background 1'

Scenario: test1
    * print 'this is test1'
    * call read('test2.feature')
    * call read('test3.feature')

test2.feature:

Feature:

Background:
    * print 'this is a background 2'
    * def testId = 222
    * configure afterScenario = function(){ karate.write({ id: testId, errorMessage: karate.info.errorMessage }, 'test-id-' + testId + '.json'); }

Scenario: test2
    * print 'this is test2'

test3.feature:

Feature:

Background:
    * print 'this is a background 3'
    * def testId = 333
    * configure afterScenario = function(){ karate.write({ id: testId, errorMessage: karate.info.errorMessage }, 'test-id-' + testId + '.json'); }

Scenario: test3
    * print 'this is test3'

I realize that writing to a file is not recommended, but in this particular case, I need to capture the test result for post execution processing (long story!). I'm also aware that the Background gets loaded for every Feature/Scenario. So once the calling feature (test1.feature) calls the first called feature (test2.feature), its Background takes over. And this is where I'd like the afterScenario hook from the called feature to be invoked, but since the calling feature (test1.feature) has not finished, it then proceeds to call the last feature (test3.feature). Similarly the Background from the the last called feature (test3.feature) gets loaded and since the calling feature (test1.feature) is now done, it then invokes the afterScenario hook from the last called feature (test3.feature), and as a result only test-id-333.json file gets written to the target folder. Here's the print output:

23:53:49.613 [ForkJoinPool-1-worker-1] INFO  com.intuit.karate - [print] this is a background 1
23:53:49.617 [ForkJoinPool-1-worker-1] INFO  com.intuit.karate - [print] this is test1
23:53:49.631 [ForkJoinPool-1-worker-1] INFO  com.intuit.karate - [print] this is a background 2
23:53:49.637 [ForkJoinPool-1-worker-1] INFO  com.intuit.karate - [print] this is test2
23:53:49.652 [ForkJoinPool-1-worker-1] INFO  com.intuit.karate - [print] this is a background 3
23:53:49.654 [ForkJoinPool-1-worker-1] INFO  com.intuit.karate - [print] this is test3

I know that this is the expected behavior, but was wondering if there is a way to invoke the afterScenario hook in the called feature (test2.feature), even if the calling feature (test1.feature) has still work to do. If I turn the test1.feature into a Scenario Outline, then both files test-id-222.json and test-id-333.json get written, since now they're each separate scenario, but unfortunately this option is not viable in this use case.

It'd be nice if there was a afterThisScenario, or a parameter to pass to a called feature i.e. * call read('test2.feature') withAfterHook, that would ensure the after hook gets invoked, even if the calling feature is still not finished.

Thanks in advance for any suggestions!

Answer 1

I solved this issue as follows:

test1.feature:

Feature:

Background:
    * print 'this is a background 1'
    * def testIds = [222, 333]
    * call read('../resources/write-test-results.js') testIds

Scenario: test1
    * print 'this is test1'
    * call read('test2.feature')
    * karate.write({ id: testId, errorMessage: null }, 'test-id-' + testId + '.json')

    * call read('test3.feature')

write-test-results.js (located in test/resources):

function fn(ids) {
    for (var i = 0; i < ids.length; i++) {
        karate.write({ id: ids[i], errorMessage: 'Skipped' }, 'test-id-' + ids[i] + '.json')
    }
}

test2.feature and test3.feature remain unchanged.

Explanation:

• The "top level" feature (test1.feature) will create two files in the target directory test-id-222.json and test-id-333.json with the following contents:

{"id":222,"errorMessage":"Skipped"}
{"id":333,"errorMessage":"Skipped"}

as a result of calling the write-test-results.js function in the Background section.

• Then the next "called" feature (test2.feature) will reload the Background and if it passes, the "top level" feature will overwrite the test-id-222.json file with this line:

* karate.write({ id: testId, errorMessage: null }, 'test-id-' + testId + '.json')

• Finally, it will call the last "called" feature (test3.feature) and since its Background will take over, the afterScenario hook will overwrite the test-id-333.json file with the actual test result.

So, if test2.feature fails, the test-id-222.json file will get overwritten with the actual error, and test-id-333.json will remain unchanged and reflect the result of test3.feature, namely Skipped.

{"id":333,"errorMessage":"Skipped"}

Answer 2

That's a little tricky to design for because you can even do this in the global karate-config.js:

karate.configure('afterScenario', function(){ karate.log('after scenario') });

And we also have a rule that afterScenario applies only to the "top level" feature and will do nothing in "called" features.

So I propose 2 options.

1. We have a concept of RuntimeHook (used to be called ExecutionHook) but you need to write some Java code, and this is somewhat un-documented, intended for advanced users. But you have full control and can read runtime metadata such as the feature name, scenario name etc.

2. define a re-usable function that takes test-id as a parameter and manually call it at the end of each block of code. But I think I see your point, you probably want this to fire even if there was an error. You can submit a feature request for an onError hook which I have been considering.