CODEWITHSUNDEEP
node.jsperformanceinsertarangodb
Garrett
Garrett

Reputation: 11725

ArangoDB inserts are extremely slow

I just went from version 1.4 to 2.1.1 and noticed improvements, but still extremely slow insert rate that gets slower as more documents are inserted.

I am migrating from my old database to new and inserting 1000 at a time. Here are some logs for your perusal:

========= [26.28s]
BATCH #1 [0s]
========= [0s]
CREATED: 1000 [0s]
UPDATED: 0 [0s]
SUBTOTAL: 1000 [0s]
BATCH TIME: 26.279s [0s]
TOTAL BATCH TIME: 26.279s [0s]
AVG TIME PER BATCH: 26.279s [0s]
TOTAL TIME: 29.277s [0s]

========= [29.919s]
BATCH #2 [0.001s]
========= [0s]
CREATED: 2000 [0s]
UPDATED: 0 [0s]
SUBTOTAL: 2000 [0s]
BATCH TIME: 29.919s [0s]
TOTAL BATCH TIME: 56.198s [0s]
AVG TIME PER BATCH: 28.099s [0s]
TOTAL TIME: 61.341s [0s]

========= [33.698s]
BATCH #3 [0s]
========= [0s]
CREATED: 3000 [0s]
UPDATED: 0 [0s]
SUBTOTAL: 3000 [0s]
BATCH TIME: 33.697s [0s]
TOTAL BATCH TIME: 89.89500000000001s [0s]
AVG TIME PER BATCH: 29.965000000000003s [0s]
TOTAL TIME: 97.755s [0s]

========= [62.653s]
BATCH #10 [0s]
========= [0s]
CREATED: 10000 [0s]
UPDATED: 0 [0s]
SUBTOTAL: 10000 [0s]
BATCH TIME: 62.653s [0s]
TOTAL BATCH TIME: 438.422s [0s]
AVG TIME PER BATCH: 43.842200000000005s [0s]
TOTAL TIME: 474.096s [0s]

I'm using the ArangoDB NodeJS API and running the following code (1000 times for each batch):

db.document.create("users", user)

Is there something I'm missing here, or is ArangoDB a lost cause for any database with >1000 records?

Upvotes: 2

Views: 853

Answers (1)

user3805876
user3805876

Reputation: 11

Please take a look to the property "waitForSync" of your collection. I've made following experience with waitForSync, creating 1000 documents and using the nodejs driver for arango:

waitForSync  true 13,3 s

waitForSync  false 0,5 s

Upvotes: 1

Related Questions