Ways to count the number of edges in Titan graph

Question

I want to count the number of edges in a graph I have loaded into Titan by means of a gremlin script. The graph is pretty small and is supposed to have less than a million edges, but when I try to do the count with g.E.count() I get the following error:

WARN com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx - Query requires iterating over all vertices [()]. For better performance, use indexes.

Permanent exception during backend operation

Stack trace:

com.thinkaurelius.titan.core.TitanException: Permanent exception during backend operation at
  com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:65) at
  com.thinkaurelius.titan.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:380) at
  com.thinkaurelius.titan.diskstorage.BackendTransaction.edgeStoreKeys(BackendTransaction.java:306) at
  com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.getVertexIDs(StandardTitanGraph.java:180) at
  com.thinkaurelius.titan.graphdb.transaction.VertexIterable$1.<init>(VertexIterable.java:31) at
  com.thinkaurelius.titan.graphdb.transaction.VertexIterable.iterator(VertexIterable.java:29) at
  com.thinkaurelius.titan.graphdb.util.VertexCentricEdgeIterable.iterator(VertexCentricEdgeIterable.java:27) at
  com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx$5.execute(StandardTitanTx.java:928) at
  com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx$5.execute(StandardTitanTx.java:788) at
  com.thinkaurelius.titan.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:231) at
  com.thinkaurelius.titan.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:32) at
  com.google.common.collect.Iterators$7.computeNext(Iterators.java:700) at
  com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) at
  com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) at
  com.thinkaurelius.titan.graphdb.query.QueryProcessor$OuterIterator.nextInternal(QueryProcessor.java:75) at
  com.thinkaurelius.titan.graphdb.query.QueryProcessor$OuterIterator.<init>(QueryProcessor.java:65) at
  com.thinkaurelius.titan.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:46) at com.google.common.collect.Iterables$7.iterator(Iterables.java:613) at
  com.tinkerpop.pipes.transform.GraphQueryPipe.processNextStart(GraphQueryPipe.java:49) at
  com.tinkerpop.pipes.transform.GraphQueryPipe.processNextStart(GraphQueryPipe.java:17) at com.tinkerpop.pipes.AbstractPipe.next(AbstractPipe.java:89) at
  com.tinkerpop.pipes.util.Pipeline.next(Pipeline.java:115) at
  com.tinkerpop.pipes.util.PipeHelper.counter(PipeHelper.java:107) at
  com.tinkerpop.gremlin.java.GremlinPipeline.count(GremlinPipeline.java:1534) at
  com.tinkerpop.gremlin.java.GremlinFluentPipeline$count.call(Unknown Source) at
  org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42) at
  org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108) at
  org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112) at groovysh_evaluate.run(groovysh_evaluate:56) at
  groovysh_evaluate$run.call(Unknown Source) at
  org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42) at groovysh_evaluate$run.call(Unknown Source) at
  org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:67) at org.codehaus.groovy.tools.shell.Interpreter$evaluate.call(Unknown Source) at
  org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:152) at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:114) at org.codehaus.groovy.tools.shell.Shell$leftShift$0.call(Unknown Source) at
  org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:88) at
  org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
  sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at
  sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at
  org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233) at
  groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1079) at
  org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:128) at
  org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148) at
  org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
  sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at
  sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at
  org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:272) at
  org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:52) at
  org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:137) at
  org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:57) at
  org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
  sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at
  sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at
  org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90) at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233) at
  groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1079) at
  org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:128) at
  org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:148) at
  org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:66) at
  com.thinkaurelius.titan.tinkerpop.gremlin.Console.<init>(Console.java:57) at
  com.thinkaurelius.titan.tinkerpop.gremlin.Console.<init>(Console.java:70) at
  com.thinkaurelius.titan.tinkerpop.gremlin.Console.main(Console.java:96)
Caused by:
  com.thinkaurelius.titan.diskstorage.PermanentStorageException: Permanent failure in storage backend at
  com.thinkaurelius.titan.diskstorage.berkeleyje.BerkeleyJEKeyValueStore.getSlice(BerkeleyJEKeyValueStore.java:162) at
  com.thinkaurelius.titan.diskstorage.keycolumnvalue.keyvalue.OrderedKeyValueStoreAdapter.getKeys(OrderedKeyValueStoreAdapter.java:88) at
  com.thinkaurelius.titan.diskstorage.keycolumnvalue.CachedKeyColumnValueStore.getKeys(CachedKeyColumnValueStore.java:134) at
  com.thinkaurelius.titan.diskstorage.BackendTransaction$5.call(BackendTransaction.java:309) at
  com.thinkaurelius.titan.diskstorage.BackendTransaction$5.call(BackendTransaction.java:306) at
  com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:62)
    ... 70 more
Caused by: com.sleepycat.je.EnvironmentFailureException:
  (JE 5.0.73) JAVA_ERROR: Java Error occurred, recovery may not be possible. at
  com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1507) at com.sleepycat.je.Database.checkEnv(Database.java:1884) at
  com.sleepycat.je.Database.openCursor(Database.java:676) at
  com.thinkaurelius.titan.diskstorage.berkeleyje.BerkeleyJEKeyValueStore.getSlice(BerkeleyJEKeyValueStore.java:119)
    ... 75 more
Caused by: java.lang.OutOfMemoryError: Java heap space at com.sleepycat.je.log.LogUtils.readBytesNoLength(LogUtils.java:365) at
  com.sleepycat.je.log.entry.LNLogEntry.readBaseLNEntry(LNLogEntry.java:205) at
  com.sleepycat.je.log.entry.LNLogEntry.readEntry(LNLogEntry.java:130) at
  com.sleepycat.je.log.LogManager.getLogEntryFromLogSource(LogManager.java:1008) at com.sleepycat.je.log.LogManager.getLogEntry(LogManager.java:848) at
  com.sleepycat.je.log.LogManager.getLogEntryAllowInvisibleAtRecovery(LogManager.java:809) at com.sleepycat.je.tree.IN.fetchTarget(IN.java:1412) at
  com.sleepycat.je.tree.BIN.fetchTarget(BIN.java:1251) at
  com.sleepycat.je.dbi.CursorImpl.fetchCurrent(CursorImpl.java:2261) at
  com.sleepycat.je.dbi.CursorImpl.getCurrentAlreadyLatched(CursorImpl.java:1466) at com.sleepycat.je.dbi.CursorImpl.getNext(CursorImpl.java:1593) at
  com.sleepycat.je.Cursor.retrieveNextAllowPhantoms(Cursor.java:2924) at com.sleepycat.je.Cursor.retrieveNextNoDups(Cursor.java:2801) at
  com.sleepycat.je.Cursor.retrieveNext(Cursor.java:2775) at
  com.sleepycat.je.Cursor.getNext(Cursor.java:1128) at
  com.thinkaurelius.titan.diskstorage.berkeleyje.BerkeleyJEKeyValueStore.getSlice(BerkeleyJEKeyValueStore.java:135) at
  com.thinkaurelius.titan.diskstorage.keycolumnvalue.keyvalue.OrderedKeyValueStoreAdapter.getSlice(OrderedKeyValueStoreAdapter.java:63) at
  com.thinkaurelius.titan.diskstorage.keycolumnvalue.CachedKeyColumnValueStore.getSlice(CachedKeyColumnValueStore.java:105) at
  com.thinkaurelius.titan.diskstorage.BackendTransaction$1.call(BackendTransaction.java:181) at
  com.thinkaurelius.titan.diskstorage.BackendTransaction$1.call(BackendTransaction.java:178) at
  com.thinkaurelius.titan.diskstorage.util.BackendOperation.execute(BackendOperation.java:62) at
  com.thinkaurelius.titan.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:380) at
  com.thinkaurelius.titan.diskstorage.BackendTransaction.edgeStoreQuery(BackendTransaction.java:178) at
  com.thinkaurelius.titan.graphdb.database.cache.PassThroughStoreCache.query(PassThroughStoreCache.java:19) at
  com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.edgeQuery(StandardTitanGraph.java:209) at
  com.thinkaurelius.titan.graphdb.query.SimpleVertexQueryProcessor$6.get(SimpleVertexQueryProcessor.java:179) at
  com.thinkaurelius.titan.graphdb.query.SimpleVertexQueryProcessor$6.get(SimpleVertexQueryProcessor.java:176) at
  com.thinkaurelius.titan.graphdb.vertices.CacheVertex.loadRelations(CacheVertex.java:39) at
  com.thinkaurelius.titan.graphdb.query.SimpleVertexQueryProcessor.getBasicIterator(SimpleVertexQueryProcessor.java:176) at
  com.thinkaurelius.titan.graphdb.query.SimpleVertexQueryProcessor.iterator(SimpleVertexQueryProcessor.java:105) at com.google.common.collect.Iterables$8.iterator(Iterables.java:713) at
  com.thinkaurelius.titan.graphdb.util.VertexCentricEdgeIterable$EdgeIterator.getNextEdge(VertexCentricEdgeIterable.

So I am not able to count the number of edges accurately. Is there an alternate method?

Answer 1

That's not an error. It is a warning. If your graph is small enough to be able to iterate E then feel free to do so. Sometimes that message is welcome as it reminds you that you are doing something possibly crazy. You should be able to tune the log level to get rid of that message.