CODEWITHSUNDEEP
karate
hueller
hueller

Reputation: 324

Scope of karate.callSingle() when having multiple Runner executions

During the upgrade of an automation project to the latest Karate version, we noticed a different behaviour of karate.callSingle() in the Karate version 1.2.0 compared to the old Karate version 0.9.6.

According to the documentation, karate.callSingle() ensures that a routine "runs only once". However, there is no clear statement about the scope in which the statement "runs only once" is true, so I don't really know if the old behaviour (in Karate 0.9.6) just worked by accident - or if there is a bug in the Karate 1.2.0.

This is our scenario:

We run our tests as JUnit 5 Parallel Execution and use the Cucumber-Reporting test report, so our test class looks very similar to https://github.com/karatelabs/karate/blob/master/karate-demo/src/test/java/demo/DemoTestParallel.java . There is one exception: When creating the Cucumber-Reporting ReportBuilder object, we add so called classifications (which are key-value pairs) to the Configuration object, because we want to display some information (artefact name, version number) about the software under test in the report. To read out the version number, we're calling a feature file, that returns the version.

As a result, the unit tests contains two Runner executions: Runner.path().parallel() to execute all tests - and Runner.runFeature() to execute only the feature that returns the version information, so that it can be passed to the Cucumber-Report:

class DemoTestParallel {

    @Test
    public void testParallel() {
        Results results = Runner.path("classpath:demo")
                .outputCucumberJson(true)
                .karateEnv("demo")
                .parallel(5);
        generateReport(results.getReportDir());
        assertTrue(results.getErrorMessages(), results.getFailCount() == 0);
    }

    public static void generateReport(String karateOutputPath) {
        Collection<File> jsonFiles = FileUtils.listFiles(new File(karateOutputPath), new String[] {"json"}, true);
        List<String> jsonPaths = new ArrayList<>(jsonFiles.size());
        jsonFiles.forEach(file -> jsonPaths.add(file.getAbsolutePath()));

        final Map<String, Object> versionResult = Runner.runFeature("classpath:demo/versionInformation.feature", null, true);

        Configuration config = new Configuration(new File("target"), "demo");
        config.addClassifications("demoArtefact", versionResult.get("version"));

        ReportBuilder reportBuilder = new ReportBuilder(jsonPaths, config);
        reportBuilder.generateReports();
    }

}

In karate-config.js, we execute a feature with karate.callSingle(), as we want to run it only once:

var order = karate.callSingle('classpath:demo/reusableOrder.feature@createSuccessfulOrder', config);
config.orderId = order.id;

In Karate version 0.9.6, karate.callSingle() was also ensured during multiple Runner executions. So in the scenario above, reusableOrder.feature@createSuccessfulOrder was executed only once.

In Karate versions 1.2.0, karate.callSingle() is not ensured during multiple Runner executions. So in the scenario above, reusableOrder.feature@createSuccessfulOrder gets executed twice (once during Runner.path().parallel() and another time during Runner.runFeature()).

Is it a bug or a feature?

As I mentioned, I don't know if this is a bug or a feature :)

In case it's a feature, I would be happy about suggestions how to avoid to have reusableOrder.feature@createSuccessfulOrder executed twice.

What I had in mind but didn't work in our case:

I am happy about any help!

Upvotes: 1

Views: 694

Answers (1)

Peter Thomas
Peter Thomas

Reputation: 58058

There's a lot to unpack in your question, but just skimming - yes the "scoping" of callSingle() to a Runner instance is intentional. The reason is it should be possible for teams to run 2 Runner instances in parallel without blocking.

So my suggestion is please stuff everything you have into a single Runner. If it means refactoring your tests, I think that's the right approach. For example, what I would do is have reusableOrder.feature write some file to the file-system, which you can then read in your reporting routine.

No other suggestions come to mind other than the extreme option of writing your own Java utility that wraps reusableOrder.feature - but ensures it returns data only once, so build your own cache within the active JVM.

Upvotes: 1

Related Questions