CODEWITHSUNDEEP
gremlintinkerpopjanusgraphtinkerpop3gremlin-server
nikmark
nikmark

Reputation: 23

Gremlin query not working after JanusGraph upgrade from 0.2.0 to 0.5.1

I upgraded JanusGraph from 0.2.0 to 0.5.1. I upgraded accordingly all the relative dependencies used in my project, like Tinkerpop (from 3.2.6 to 3.4.6 as requested by JanusGraph). I followed all the upgrade guides of Tinkerpop and JanusGraph and everything seems to work well except some Gremlin queries that generate ClassCastException without any further stack information. The same exact queries work on JanusGraph 0.2.0 smoothly. One of the incriminated queries is the following one:

g.V(entry).match(
 __.as('n').properties().has('to', -1).hasNot('stage').as('p'),
 __.as('p').label().as('label'),
 __.as('p').values('from').as('from'),
 __.as('n').properties().has('to', -1).hasNot('stage')
   .where(eq('p')).by(T.label)
   .values('from').max().as('max')
 ).where('from', neq('max'))
  .select('p')
  .property('to', __.select('max'))
  .iterate();

The stack trace is the following:

java.lang.ClassCastException
Type ':help' or ':h' for help.
Display stack trace? [yN]y
java.lang.ClassCastException
gremlin>

No information. I think I'm missing something related to some version upgrades but from the Tinkerpop upgrade documentation, I could not find any insight.

The entry on which I'm working is the following:

gremlin> g.V(entry).elementMap()
==>{id=5533792, label=entry, uri=test_uri, _metadata=1591698356515, _last_stage=published, _last_from=1591698356515, type=person, id=test_id} 

gremlin> g.V(entry).valueMap(true)
==>{id=5533792, label=entry, uri=[test_uri], _metadata=[1591698356939, 1591698356515], _last_stage=[published, published], _last_from=[1591698356939, 1591698356515], type=[person], id=[test_id]}

gremlin> g.V(entry).properties()
==>vp[uri->test_uri]
==>vp[_metadata->1591698356939]
==>vp[_metadata->1591698356515]
==>vp[_last_stage->published]
==>vp[_last_stage->published]
==>vp[_last_from->1591698356939]
==>vp[_last_from->1591698356515]
==>vp[type->person]
==>vp[id->test_id]

Any ideas on what I'm missing?? Thank you in advance.

EDIT 1 As @stephenmallette suggested, I started to reduce the statement. Start point for comfort was the following:

g.V(entry).match(
 __.as('n').properties().has('to', -1).hasNot('stage').as('p'),
 __.as('p').label().as('label'),
 __.as('p').values('from').as('from'),
 __.as('n').properties().has('to', -1).hasNot('stage')
   .where(eq('p')).by(T.label)
   .values('from').max().as('max')
 ).where('from', neq('max')).select('p')

Still exception with no stack trace. I removed in order:

g.V(entry).match(
__.as('n').properties().has('to', -1).hasNot('stage').as('p'),
__.as('p').label().as('label'),
__.as('p').values('from').as('from'),
__.as('n').properties().has('to', -1).hasNot('stage')
  .where(eq('p')).by(T.label)
  .values('from').max().as('max')
).select('p')

Result: ClassCastException with no stack trace

g.V(entry).match(
__.as('n').properties().has('to', -1).hasNot('stage').as('p'),
__.as('p').label().as('label'),
__.as('p').values('from').as('from')
).select('p')

Result: ClassCastException with no stack trace

g.V(entry).match(
__.as('n').properties().has('to', -1).hasNot('stage').as('p'),
__.as('p').label().as('label')
).select('p')

Result: we have a winner. Result as following:

==>vp[_last_stage->published]
==>vp[_last_stage->published]
==>vp[_last_from->1591698356939]
==>vp[_last_from->1591698356515]

So I tried something more. I restored the last removed traversal of the match (__as('p').values('from').as('from')) and removed the last select('p') like the following:

g.V(entry).match(
__.as('n').properties().has('to', -1).hasNot('stage').as('p'),
__.as('p').label().as('label'),
__.as('p').values('from').as('from')
)

That's the result:

==>{p=vp[_last_stage->published], from=1591698356939, label=_last_stage, n=v[5533792]}
==>{p=vp[_last_stage->published], from=1591698356515, label=_last_stage, n=v[5533792]}
==>{p=vp[_last_from->1591698356939], from=1591698356939, label=_last_from, n=v[5533792]}
==>{p=vp[_last_from->1591698356515], from=1591698356515, label=_last_from, n=v[5533792]}

Now I'm even more confused than before. What can I do? Thank you in advance.

EDIT 2

I ran again the query today after a stack restart (JanusGraph is a container in a swarm stack) and this is the result now:

gremlin> g.V(entry).match(
,.....1>           __.as("n").properties().has("to", -1).hasNot("stage").as("p")
......2>           __.as("p").label().as("label"),
......3>           __.as("p").values("to").as("from"),
......4>           __.as("n").properties().has("to", -1).hasNot("stage")
......5>               .where(eq("p")).by(T.label)
......6>               .values("from").max().as("max")
......7>           ).where("from", neq("max")
......8>           ).select('p')
org.janusgraph.graphdb.relations.CacheVertexProperty cannot be cast to org.apache.tinkerpop.gremlin.structure.Vertex
Type ':help' or ':h' for help.
Display stack trace? [yN]y
java.lang.ClassCastException: org.janusgraph.graphdb.relations.CacheVertexProperty cannot be cast to org.apache.tinkerpop.gremlin.structure.Vertex
    at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.lambda$initializeMultiQuery$1(JanusGraphPropertiesStep.java:110)
    at java.util.Collections$SingletonList.forEach(Collections.java:4824)
    at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.initializeMultiQuery(JanusGraphPropertiesStep.java:110)
    at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.flatMap(JanusGraphPropertiesStep.java:132)
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:49)
    at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.processNextStart(JanusGraphPropertiesStep.java:125)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.MatchStep$MatchEndStep.processNextStart(MatchStep.java:569)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
    at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:197)
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.MatchStep.standardAlgorithm(MatchStep.java:371)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.ComputerAwareStep.processNextStart(ComputerAwareStep.java:46)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:36)
    at org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectOneStep.processNextStart(SelectOneStep.java:131)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
    at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:197)
    at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.handleIterator(AbstractOpProcessor.java:93)
    at org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor.lambda$evalOpInternal$5(AbstractEvalOpProcessor.java:264)
    at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:278)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

I am even more confused. I cannot find an alternative way to perform my query, and I cannot find what is the problem in it. Can someone help me? Thanks

EDIT 3 Hi all and @stephenmallette. I had the chance to continue my analysis and I think I found a bug. I decided to restart from the beginning and create a sample graph that recreates this situation.

Creation of test node:

if(g.tx().isOpen()) { g.tx().rollback() } 
g.tx().open()

v = [:]
v["test"]=graph.addVertex('test_vertex')         
v["test"].property("uri", "test")
v["test"].property("type", "test_type")
v["test"].property("id", "123")
v["test"].property("test_prop", 1593189100420,"from",1593189100420,"to",-1)
v["test"].property("test_prop", 1593189099710,"from",1593189099710,"to",-1)
g.tx().commit()

Adapted query:

g.V().has("uri", "test").match(
     __.as('n').properties().has('to', -1).as('p'),
     __.as('p').label().as('label'),
     __.as('p').values('from').as('from'),
     __.as('n').properties().has('to', -1)
         .where(eq('p')).by(T.label)
         .values('from').max().as('max')
     ).where('from', neq('max')).select('p').property('to', __.select('max')).iterate()

Exception:

67764 [gremlin-server-session-1] WARN  org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor  - Exception processing a script on request [RequestMessage{, requestId=4cc4de8c-916c-487d-9696-994e283fb8a7, op='eval', processor='session', args={gremlin=g.V().has("uri", "test").match(
     __.as('n').properties().has('to', -1).as('p'),
     __.as('p').label().as('label'),
     __.as('p').values('from').as('from'),
     __.as('n').properties().has('to', -1)
         .where(eq('p')).by(T.label)
         .values('from').max().as('max')
     ).where('from', neq('max')).select('p').property('to', __.select('max')).iterate(); session=0dd79d9d-1bb5-4fb0-8400-027667ac236f, manageTransaction=false, userAgent=Gremlin Console/3.4.6, batchSize=64}}].
java.lang.ClassCastException: org.janusgraph.graphdb.relations.CacheVertexProperty cannot be cast to org.apache.tinkerpop.gremlin.structure.Vertex
at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.lambda$initializeMultiQuery$1(JanusGraphPropertiesStep.java:110)
at java.util.Collections$SingletonList.forEach(Collections.java:4824)
at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.initializeMultiQuery(JanusGraphPropertiesStep.java:110)
at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.flatMap(JanusGraphPropertiesStep.java:132)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:49)
at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphPropertiesStep.processNextStart(JanusGraphPropertiesStep.java:125)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.MatchStep$MatchEndStep.processNextStart(MatchStep.java:569)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:197)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.MatchStep.standardAlgorithm(MatchStep.java:371)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ComputerAwareStep.processNextStart(ComputerAwareStep.java:46)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:36)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectOneStep.processNextStart(SelectOneStep.java:131)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.hasNext(ExpandableStepIterator.java:42)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.NoOpBarrierStep.processAllStarts(NoOpBarrierStep.java:66)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.NoOpBarrierStep.processNextStart(NoOpBarrierStep.java:55)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
at org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SideEffectStep.processNextStart(SideEffectStep.java:38)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
at org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep.processNextStart(FilterStep.java:37)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38)
at org.apache.tinkerpop.gremlin.process.traversal.Traversal.iterate(Traversal.java:207)
at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal.iterate(GraphTraversal.java:2931)
at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal$Admin.iterate(GraphTraversal.java:189)
at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.DefaultGraphTraversal.iterate(DefaultGraphTraversal.java:48)
at org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal$Admin$iterate.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:115)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:119)
at Script14.run(Script14.groovy:8)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:674)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:376)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:267)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

I decided so to avoid using my conf, and use the Janusgraph latest docker image

docker run --name janusgraph-default janusgraph/janusgraph:latest

I created again my test node and executed again the query and finally no errors! So I compared the gremlin-server.yml and janusgraph.properties file (mine and the default ones). Found (of course) some differences so I tried to exclude one by one the differences. The property that "creates" this problem resulted to be the query-batch=true that I added in janusgraph.properties.

So I run the default image like that

docker run -e janusgraph.query.batch=true --name janusgraph-default janusgraph/janusgraph:latest

create the sample node and executed the query et voilà, exception raised. Removing it everything works like a charm

I looked in the Jira issues of Janusgraph and i found some issues related to multiquery optimization #71 #1000 PR #1000, but they are already solved.

Since the query-batch parameter should be an optimization, I found odd that an optimization broke everything somehow. How can I address this? Any idea?

Thank you in advance again.

Upvotes: 1

Views: 682

Answers (1)

stephen mallette
stephen mallette

Reputation: 46216

I can verify that your traversal works in TinkerGraph so I suspect that you've found a bug in JanusGraph:

gremlin> g = TinkerGraph.open().traversal()
==>graphtraversalsource[tinkergraph[vertices:0 edges:0], standard]
gremlin> g.addV("test_vertex").property("uri", "test").property("type", "test_type").property("id", "123").
......1>                       property(list,"test_prop", 1593189100420,"from",1593189100420,"to",-1).
......2>                       property(list,"test_prop", 1593189099710,"from",1593189099710,"to",-1)
==>v[0]
gremlin> g.V().properties('test_prop').valueMap()
==>[from:1593189100420,to:-1]
==>[from:1593189099710,to:-1]
gremlin> g.V().has("uri", "test").match(
......1>      __.as('n').properties().has('to', -1).as('p'),
......2>      __.as('p').label().as('label'),
......3>      __.as('p').values('from').as('from'),
......4>      __.as('n').properties().has('to', -1)
......5>          .where(eq('p')).by(T.label)
......6>          .values('from').max().as('max')
......7>      ).where('from', neq('max')).select('p').property('to', __.select('max'))
==>vp[test_prop->1593189099710]
gremlin> g.V().properties('test_prop').valueMap()
==>[from:1593189100420,to:-1]
==>[from:1593189099710,to:1593189100420]

I unfortunately don't know what would cause that problem. If you have not done so already I think you should submit these reproduction steps to the JanusGraph project.

Upvotes: 1

Related Questions