CODEWITHSUNDEEP
lucenesparqlrdfrdf4j
user2917629
user2917629

Reputation: 942

RDF / lucenesail error: org.openrdf.query.QueryEvaluationException: java.util.ConcurrentModificationException

We use http://www.openrdf.org/contrib/lucenesail# search in 90% of our rdf queries. I'd like to see if anyone knows how to go about fixing the error "org.openrdf.query.QueryEvaluationException: java.util.ConcurrentModificationException" in file /sesame-data/openrdf-sesame/logs/main.log. When the error occurs, it is closely followed by rapidly elevating rdf server CPU usage, rdf server unresponsiveness and a necessity to restart the server. It appears to happen at random, unrelated to a particular query. I wonder if the error may be related to the use of lucenesail interactions with rdf data since we haven't seen these errors until after converting our queries to lucenesail queries. Will appreciate your input. Thank you for your time. This is a full error trace:

    [ERROR] 2016-09-08 10:07:38,826 [repositories/NwsRDFRepo] LuceneTripleSource: Error getting statements for http://acfews.com/content/41806958 http://acfews.com/content/type "Image"^^<htt
    p://www.w3.org/2001/XMLSchema#token>java.util.ConcurrentModificationException: null
            at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:886)
            at java.util.ArrayList$Itr.next(ArrayList.java:836)
            at org.openrdf.model.impl.TreeModel.choose(TreeModel.java:474)
            at org.openrdf.model.impl.TreeModel.matchPattern(TreeModel.java:406)
            at org.openrdf.model.impl.TreeModel.contains(TreeModel.java:250)
            at org.openrdf.sail.nativerdf.MemoryOverflowModel.contains(MemoryOverflowModel.java:127)
            at org.openrdf.model.impl.FilteredModel.contains(FilteredModel.java:157)
            at org.openrdf.model.impl.AbstractModel.isEmpty(AbstractModel.java:58)
            at org.openrdf.sail.base.SailDatasetImpl.difference(SailDatasetImpl.java:286)
            at org.openrdf.sail.base.SailDatasetImpl.getStatements(SailDatasetImpl.java:265)
            at org.openrdf.sail.base.DelegatingSailDataset.getStatements(DelegatingSailDataset.java:79)
            at org.openrdf.sail.base.DelegatingSailDataset.getStatements(DelegatingSailDataset.java:79)
            at org.openrdf.sail.base.UnionSailDataset.getStatements(UnionSailDataset.java:111)
            at org.openrdf.sail.base.SailSourceConnection.getStatementsInternal(SailSourceConnection.java:258)
            at org.openrdf.sail.helpers.SailConnectionBase.getStatements(SailConnectionBase.java:328)
            at org.openrdf.sail.lucene.LuceneTripleSource.getStatements(LuceneTripleSource.java:284)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:517)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:226)
            at org.openrdf.query.algebra.evaluation.iterator.JoinIterator.<init>(JoinIterator.java:54)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:861)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:824)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:232)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.<init>(LeftJoinIterator.java:65)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:883)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:827)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:232)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.<init>(LeftJoinIterator.java:65)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:883)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:827)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:232)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:88)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:33)
            at info.aduna.iteration.LookAheadIteration.lookAhead(LookAheadIteration.java:91)
            at info.aduna.iteration.LookAheadIteration.hasNext(LookAheadIteration.java:60)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:80)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:33)
            at info.aduna.iteration.LookAheadIteration.lookAhead(LookAheadIteration.java:91)
            at info.aduna.iteration.LookAheadIteration.hasNext(LookAheadIteration.java:60)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:80)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:33)
            at info.aduna.iteration.LookAheadIteration.lookAhead(LookAheadIteration.java:91)
            at info.aduna.iteration.LookAheadIteration.hasNext(LookAheadIteration.java:60)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:80)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:33)
            at info.aduna.iteration.LookAheadIteration.lookAhead(LookAheadIteration.java:91)
            at info.aduna.iteration.LookAheadIteration.hasNext(LookAheadIteration.java:60)
            at info.aduna.iteration.ConvertingIteration.hasNext(ConvertingIteration.java:73)
            at info.aduna.iteration.ConvertingIteration.hasNext(ConvertingIteration.java:73)
            at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68)
            at info.aduna.iteration.FilterIteration.findNextElement(FilterIteration.java:79)
            at info.aduna.iteration.FilterIteration.hasNext(FilterIteration.java:54)
            at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68)
            at info.aduna.iteration.LimitIteration.hasNext(LimitIteration.java:73)
            at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68)
            at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68)
            at org.openrdf.query.QueryResults.report(QueryResults.java:155)
            at org.openrdf.http.server.repository.TupleQueryResultView.renderInternal(TupleQueryResultView.java:110)
            at org.openrdf.http.server.repository.QueryResultView.render(QueryResultView.java:84)
            at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1228)
            at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1011)
            at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:955)
            at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877)
            at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
            at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
            at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
            at com.github.ziplet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:300)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
            at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
            at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
            at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
            at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
            at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)

    [ERROR] 2016-09-08 10:07:38,827 [repositories/NwsRDFRepo] Query evaluation error
    org.openrdf.query.QueryEvaluationException: java.util.ConcurrentModificationException
            at org.openrdf.sail.lucene.LuceneTripleSource.getStatements(LuceneTripleSource.java:294)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:517)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:226)
            at org.openrdf.query.algebra.evaluation.iterator.JoinIterator.<init>(JoinIterator.java:54)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:861)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:824)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:232)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.<init>(LeftJoinIterator.java:65)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:883)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:827)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:232)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.<init>(LeftJoinIterator.java:65)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:883)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:827)
            at org.openrdf.query.algebra.evaluation.impl.EvaluationStrategyImpl.evaluate(EvaluationStrategyImpl.java:232)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:88)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:33)
            at info.aduna.iteration.LookAheadIteration.lookAhead(LookAheadIteration.java:91)
            at info.aduna.iteration.LookAheadIteration.hasNext(LookAheadIteration.java:60)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:80)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:33)
            at info.aduna.iteration.LookAheadIteration.lookAhead(LookAheadIteration.java:91)
            at info.aduna.iteration.LookAheadIteration.hasNext(LookAheadIteration.java:60)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:80)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:33)
            at info.aduna.iteration.LookAheadIteration.lookAhead(LookAheadIteration.java:91)
            at info.aduna.iteration.LookAheadIteration.hasNext(LookAheadIteration.java:60)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:80)
            at org.openrdf.query.algebra.evaluation.iterator.LeftJoinIterator.getNextElement(LeftJoinIterator.java:33)
            at info.aduna.iteration.LookAheadIteration.lookAhead(LookAheadIteration.java:91)
            at info.aduna.iteration.LookAheadIteration.hasNext(LookAheadIteration.java:60)
            at info.aduna.iteration.ConvertingIteration.hasNext(ConvertingIteration.java:73)
            at info.aduna.iteration.ConvertingIteration.hasNext(ConvertingIteration.java:73)
            at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68)
            at info.aduna.iteration.FilterIteration.findNextElement(FilterIteration.java:79)
            at info.aduna.iteration.FilterIteration.hasNext(FilterIteration.java:54)
            at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68)
            at info.aduna.iteration.LimitIteration.hasNext(LimitIteration.java:73)
            at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68)
            at info.aduna.iteration.IterationWrapper.hasNext(IterationWrapper.java:68)
            at org.openrdf.query.QueryResults.report(QueryResults.java:155)
            at org.openrdf.http.server.repository.TupleQueryResultView.renderInternal(TupleQueryResultView.java:110)
            at org.openrdf.http.server.repository.QueryResultView.render(QueryResultView.java:84)
            at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1228)
            at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1011)
            at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:955)
            at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877)
            at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
            at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
            at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
            at com.github.ziplet.filter.compression.CompressingFilter.doFilter(CompressingFilter.java:300)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
            at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
            at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
            at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
            at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
            at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    Caused by: java.util.ConcurrentModificationException: null
            at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:886)
            at java.util.ArrayList$Itr.next(ArrayList.java:836)
            at org.openrdf.model.impl.TreeModel.choose(TreeModel.java:474)
            at org.openrdf.model.impl.TreeModel.matchPattern(TreeModel.java:406)
            at org.openrdf.model.impl.TreeModel.contains(TreeModel.java:250)
            at org.openrdf.sail.nativerdf.MemoryOverflowModel.contains(MemoryOverflowModel.java:127)
            at org.openrdf.model.impl.FilteredModel.contains(FilteredModel.java:157)
            at org.openrdf.model.impl.AbstractModel.isEmpty(AbstractModel.java:58)
            at org.openrdf.sail.base.SailDatasetImpl.difference(SailDatasetImpl.java:286)
            at org.openrdf.sail.base.SailDatasetImpl.getStatements(SailDatasetImpl.java:265)
            at org.openrdf.sail.base.DelegatingSailDataset.getStatements(DelegatingSailDataset.java:79)
            at org.openrdf.sail.base.DelegatingSailDataset.getStatements(DelegatingSailDataset.java:79)
            at org.openrdf.sail.base.UnionSailDataset.getStatements(UnionSailDataset.java:111)
            at org.openrdf.sail.base.SailSourceConnection.getStatementsInternal(SailSourceConnection.java:258)
            at org.openrdf.sail.helpers.SailConnectionBase.getStatements(SailConnectionBase.java:328)
            at org.openrdf.sail.lucene.LuceneTripleSource.getStatements(LuceneTripleSource.java:284)
            ... 70 common frames omitted

Upvotes: 1

Views: 146

Answers (1)

Jeen Broekstra
Jeen Broekstra

Reputation: 22042

I haven't seen this particular error, but the version of Lucene Sail you're using looks like it is very old - back when it was still a separate project.

The Lucene Sail has been integrated with RDF4J, and has been extensively improved and updated. My suggestion would be to upgrade to the latest version of RDF4J and switch to the built-in Lucene Sail. For more information on how to upgrade an existing Sesame installation to RDF4J, see the Migration Guide.

Upvotes: 1

Related Questions