CODEWITHSUNDEEP
graph-databasesorientdb
JP Moresmau
JP Moresmau

Reputation: 7403

Very slow insertion performance in OrientDB via Java API

I'm evaluating the replacement of a relational database by a graph database. I'm trying to copy the data over to OrientDB (version 2.0) from the original database. I create an embedded server via OServer server = OServerMain.create();, passing false to storage.useWAL. Then I create a non transactional graph:

OrientBaseGraph graph = new OrientGraphNoTx("plocal:"+db);
graph.declareIntent(new OIntentMassiveInsert());
graph.getRawGraph().declareIntent(new OIntentMassiveInsert());
graph.getRawGraph().setValidationEnabled(false);

I create vertex types for all my tables. I first create all vertices with their attributes in on call for each vertex:

OrientVertex node = graph.addVertex("class:"+lbl, properties);

I then create edges between these vertices. Some (but not all) of these edges have properties.

if (props!=null){
    nfrom.addEdge(linkName, nto,null,null,props);
} else {
    nfrom.addEdge(linkName, nto);
}

I've tried with and without edge classes, didn't notice any performance improvements. All in all, I have 328822 vertices and 831293 edges. The total run time is at best around 25 minutes!! Most of that time (20 minutes at least) is spent inserting the edges, not the vertices.

On the same machine, reading the same data from the same relation db and writing it in Titan with BerkeleyDB backend, I transfer the data in 2 minutes!

What makes OrientDB around 10 times slower that a competitor? What do I do wrong?

Thanks!

Upvotes: 2

Views: 2254

Answers (1)

Lvca
Lvca

Reputation: 9060

When you have many edges, it's much better using Transactional Graph and commit every X items. Furthermore disable the journal for TX. Example:

OrientBaseGraph graph = new OrientGraph("plocal:"+db);
try{
  graph.getRawGraph().getTransaction().setUsingLog(false);

  int saved = 0;
  while(){ // this is your loop
    ....

    saved++;

    if( saved % 5000 == 0 ){
      graph.commit();
      graph.getRawGraph().getTransaction().setUsingLog(false);
    }
  }
  graph.commit();

} finally {
  graph.close();
}

Upvotes: 0

Related Questions