CODEWITHSUNDEEP
gremlinamazon-neptune
Sahas
Sahas

Reputation: 3186

aws neptune times out for large graph drop ()

There are some threads on this subject already.. particularly this one

but is there any recommended solution to drop large graph other than batching..? I tried increasing timeout and it doesn't work

Below is the example..

gremlin> g.V().count()

==>5230885

gremlin> g.V().drop().iterate()

{"requestId":"77c64369-45fa-462f-91d7-5712e3308497","detailedMessage":"A timeout occurred within the script during evaluation of [RequestMessage{, requestId=77c64369-45fa-462f-91d7-5712e3308497, op='eval', processor='', args={gremlin=g.V().drop().iterate(), bindings={}, batchSize=64}}] - consider increasing the timeout","code":"TimeLimitExceededException"} Type ':help' or ':h' for help. Display stack trace? [yN]N

gremlin> g.E().count()

==>83330550

gremlin> :remote config timeout none

==>Remote timeout is disabled

gremlin> g.E().drop().iterate()

{"requestId":"d418fa03-72ce-4154-86d8-42225e4b9eca","detailedMessage":"A timeout occurred within the script during evaluation of [RequestMessage{, requestId=d418fa03-72ce-4154-86d8-42225e4b9eca, op='eval', processor='', args={gremlin=g.E().drop().iterate(), bindings={}, batchSize=64}}] - consider increasing the timeout","code":"TimeLimitExceededException"} Type ':help' or ':h' for help. Display stack trace? [yN]N

Upvotes: 4

Views: 2910

Answers (3)

Kelvin Lawrence
Kelvin Lawrence

Reputation: 14371

You have two options currently to drop an entire graph that is large. One option of course is to delete the current cluster and create a new one. To delete the existing graph the best approach is to use multiple threads that drop chunks of the graph in batches. I have been working on some Python code that can do just that. It is currently on a branch at this location.

https://github.com/awslabs/amazon-neptune-tools/tree/master/drop-graph

For a graph of the size you have the tool should work fine as is. It does have some limitations currently that are documented in the code.

UPDATED 2021-Dec-8 to add:

Since this question was asked Amazon Neptune now supports a Fast Reset API that can be used to delete all of the data in a cluster. More details here: https://docs.aws.amazon.com/neptune/latest/userguide/manage-console-fast-reset.html

Upvotes: 1

Oswin Noetzelmann
Oswin Noetzelmann

Reputation: 9545

If you have trouble with timeouts on LOAD or UNLOAD requests consider that the neptune_query_timeout configuration setting applies, but has to be set for every DB instance that you address (and not just once for the cluster). Different configuration parameter sets can be applied to the cluster and the single instances, so make sure the parameter set for the instance in question has the right timeout setting.

Upvotes: 0

Kfir Dadosh
Kfir Dadosh

Reputation: 1419

You can increase the timeout of your neptune cluster by using the parameter group option neptune_query_timeout.

If using version 3.3.7 of the Java client, you can specify it for specific requests: Set Timeouts at a Per-Query Level

Hopefully soon you will be able to run:

g.with("scriptEvaluationTimeout", 600).V().drop()

Upvotes: 3

Related Questions