CODEWITHSUNDEEP
javaxpathgroovysaxon
Volodya Lombrozo
Volodya Lombrozo

Reputation: 3466

Why is Saxon initially slow but then fast in my XPath benchmarks?

I’m benchmarking Saxon for some XPath expressions in Groovy/Java. I measure execution time by recreating Processor, DocumentBuilder, and XPathCompiler each time, hoping to get consistent "cold" runs.

static def measureSaxon(String xml, String xpath) {
    Processor processor = new Processor(false);
    processor.setConfigurationProperty(FeatureKeys.OPTIMIZATION_LEVEL, "0")
    XdmNode xdm = processor.newDocumentBuilder().build(new StreamSource(new StringReader(xml)));
    def compiler = processor.newXPathCompiler()
    return measureExecutionTime(
      "Saxon",
      xpath,
      {
          def evaluate = compiler.evaluate(xpath, xdm)
          if (evaluate instanceof XdmValue && evaluate.size() > 0) {
              return evaluate.itemAt(0).getStringValue()
          } else {
              return "No node found"
          }
      }
    )
}

However, the first invocation is always much slower than subsequent calls, which appear to benefit from some kind of caching or optimization—even though I’ve tried:

processor.setConfigurationProperty(FeatureKeys.OPTIMIZATION_LEVEL, "0")

Below are my benchmark results. You can see how the first call can be very slow, and subsequent calls are much faster:

# Benchmark Results for Saxon

| XPath Expression                | Execution Time (ns) | Execution Time (ms) | Result               |
|---------------------------------|---------------------|---------------------|----------------------|
| `/program/@name`                | 151562270           | 151.56227           | j$Collections        |
| `/program/objects/o/@base`      | 522288              | 0.522288            | some.class           |
| `/program/objects/o/o/o/o/@base`| 12328695            | 12.328695           | org.lang.bytes       |

Is there a deeper caching mechanism or an internal optimization in Saxon itself that cannot be disabled simply by resetting the Processor or setting OPTIMIZATION_LEVEL to zero? How can I force Saxon to always run as if it’s on the "first" invocation, to measure truly cold performance?

Any pointers on disabling (or working around) Saxon’s internal caching or optimizations would be a big help. Thanks!

Upvotes: 0

Views: 75

Answers (1)

Michael Kay
Michael Kay

Reputation: 163468

Unless you've taken considerable care to prevent it, this is Java warm-up time. Firstly, Java has to load the Saxon classes, and it does this incrementally as each class is first referenced. Then the hot-spot compiler monitors which methods are called most frequently, and when it detects a hot-spot it performs optimizations such as inlining, and where appropriate generation of native machine code. It typically takes a couple of seconds before everything is running at full speed.

There is also some Saxon initialization cost, such as setting up indexes of standard data types and built-in functions. Some of this is done once for Saxon as a whole, some is done once when you create a Saxon processor/configuration. But I think Saxon's initialization costs are much lower than the Java warm-up.

Obviously a lesson from this is that in production, you should avoid at all costs loading up a new Java VM for each Saxon task.

Upvotes: 2

Related Questions