Saxon xquery validation fails - help needed to understand behavior

Question

Whilst i am using Saxon java api to execute the following xQuery, i am unable to understand why the following execution / validation fails ? (Interestingly , when i replace and clause with or in the if statement, the query validation is successful but am unable to understand this behaviour )

In oxygen xml validator, when i open the xQuery, i get NullPointerException-null exception and the validation fails.

And in java , i get the following area

java.lang.NullPointerException
    at net.sf.saxon.expr.parser.LoopLifter.markDependencies(LoopLifter.java:168)
    at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:112)

I am looking up for some experts to help me understand why the following fails.

Following is the xQuery,

 
    declare function local:method($input as element(input)) as element(output) 
        {
                <output>
                    <itemADetails>              
                        <service>
                        {
                        for $i in 1 to count($input/*:foo)
                        return
                            for $j in 1 to count($input/*:bar)
                            return
                                if((data($input/*:foo[$i]/*:itemB[1]/*:rangeQualifier)="A") and (data($input/*:foo[$i]/*:serviceId/*:type)="B") ) then
                                    <node></node>
                                else()          
                        }
                        </service>                                          
                    </itemADetails>                                         
                </output>
        };


    declare variable $input as element(input) external;
    local:method($input)

Saxon Verion

implementation 'net.sf.saxon:Saxon-HE:10.2'
implementation 'net.sf.saxon:saxon-xqj:9.1.0.8'

Sample Snippet that i tried

 Processor saxon = new Processor(false);
         
        // compile the query
        XQueryCompiler compiler = saxon.newXQueryCompiler();
        XQueryExecutable exec;
       
        ClassLoader classLoader = MessageProcessor.class.getClassLoader();
       
            exec = compiler.compile(new File(classLoader.getResource("Xquery.xqy").getFile()));

Source src = new StreamSource(new StringReader(Files.readString( Paths.get(ClassLoader.getSystemResource(inputfile.xml").toURI()))));
        XdmNode doc = builder.build(src);
 // instantiate the query, bind the input and evaluate
        XQueryEvaluator query = exec.load();
        query.setContextItem(doc);     
 query.setExternalVariable(new QName("input"), doc.select(child("input")).asNode());
   XdmValue result = query.evaluate();
       System.out.println(result.itemAt(0).toString());

Irrespective of the java code when i open the xquery in Oxygen XML editor (that uses Saxon-PE XQuery 9.9.1.7 engine) , i get the following validation error.

Answer 1

It is indeed a Saxon optimisation bug. The problem occurs when Saxon attempts to rewrite

for $j in 1 to count($input/bar)
return if ($input/foo[$i]/data = 'A' and $input/baz[$i]/type = 'B')
       then <result/>
       else ()

as

if ($input/foo[$i]/data = 'A' and $input/baz[$i]/type = 'B')
then for $j in 1 to count($input/bar)
     return <result/>
else ()

and you can work around the problem by doing this rewrite "by hand". The purpose of the rewrite is to prevent the unnecessary repeated evaluation of the "if" condition, which is the same each time round the loop.

The reason it's dependent on the "and" condition is that Saxon considers each term of the "and" as a separate candidate for promoting outside the loop, and when it finds that all these terms can be promoted, it reconstitutes the "and" expression from its parts, and the bug occurs during this reconstitution.

Answer 2

I reproduced it with Martin's test case (thank you, Martin): https://saxonica.plan.io/issues/4765

Answer 3

It seems like an optimizer bug in Saxon, I reduced your code to

declare function local:test($input as element(input)) as element(output)
{
  <output>
      <details>
          <service>
          {
              for $i in 1 to count($input/foo)
              return
                  for $j in 1 to count($input/bar)
                  return if ($input/foo[$i]/data = 'A' and $input/baz[$i]/type = 'B')
                         then <result/>
                         else ()
          }
          </service>
      </details>
  </output>
};

declare variable $input as element(input) external := <input>
</input>;

local:test($input)

and Saxon HE 10.2 from the command line crashes with

java.lang.NullPointerException
        at net.sf.saxon.expr.parser.LoopLifter.markDependencies(LoopLifter.java:168)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:112)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:122)
        at net.sf.saxon.expr.parser.LoopLifter.gatherInfo(LoopLifter.java:101)
        at net.sf.saxon.expr.parser.LoopLifter.process(LoopLifter.java:51)
        at net.sf.saxon.query.XQueryFunction.optimize(XQueryFunction.java:452)
        at net.sf.saxon.query.XQueryFunctionLibrary.optimizeGlobalFunctions(XQueryFunctionLibrary.java:327)
        at net.sf.saxon.query.QueryModule.optimizeGlobalFunctions(QueryModule.java:1207)
        at net.sf.saxon.expr.instruct.Executable.fixupQueryModules(Executable.java:458)
        at net.sf.saxon.query.XQueryParser.makeXQueryExpression(XQueryParser.java:177)
        at net.sf.saxon.query.StaticQueryContext.compileQuery(StaticQueryContext.java:568)
        at net.sf.saxon.query.StaticQueryContext.compileQuery(StaticQueryContext.java:630)
        at net.sf.saxon.s9api.XQueryCompiler.compile(XQueryCompiler.java:609)
        at net.sf.saxon.Query.compileQuery(Query.java:804)
        at net.sf.saxon.Query.doQuery(Query.java:317)
        at net.sf.saxon.Query.main(Query.java:97)
java.lang.NullPointerException

I think on the command line you can turn off optimization with -opt:0, then the above code doesn't crash.

You might want to raise your bug as an issue on https://saxonica.plan.io/projects/saxon/issues or wait until someone from Saxonica picks it up here.

If I use

           for $foo at $i in $input/foo
          return
              for $bar at $j in $input/bar

Saxon doesn't crash, so perhaps that is a way to rewrite your query as a workaround, although, without data, I am not quite sure I have captured the meaning of your code and that rewrite does the same as your original attempt.