CODEWITHSUNDEEP
xodus
user2163960
user2163960

Reputation: 1911

xodus write performance with parallel threads?

I'm attempting to use xodus as a store, because I want a store of that type, and the benchmarks make it out to be fast.

However, in practice, the writes seem to be really slow.

I'm setting up two stores with a pattern like this:

environment = Environments.newInstance(xodusDBFolder);  
Store initialStateStore = environment.computeInTransaction(new TransactionalComputable<Store>()
    {
        @Override
        public Store compute(@NotNull final Transaction txn)
        {
            return environment.openStore("initialStateStore", StoreConfig.WITHOUT_DUPLICATES, txn);
        }
    });

Then, I write with a pattern like this:

environment.executeInTransaction((Transaction txn) -> {
    ArrayByteIterable key = IntegerBinding.intToEntry(c.getNid());
    ByteIterable oldValue = store.get(txn, key);    
    ArrayByteIterable value;
    if (oldValue != null)    
    {
        //merge contents...
    }
    ....
   value = new ArrayByteIterable(rawData);
   store.put(txn, key, value);
});

Feeding the data into this pattern, I have a couple of threads, each reading and parsing data - 1 for each environment that I am populating.
I had expected my read / parse to the the bottleneck... but in practice, the bottleneck is in the xodus write.

I had thought that I could have xodus write in parallel, so I did my writes in a thread pool - but that made the performance even worse. To date, the best performance seems to happen when I disable the garbage collector, and only allow 1 thread to write to the xodus store.

Am I doing something wrong? With one thread, the slow part is everything that happens below ReadWriteTransaction.doCommit(). With multiple threads, they all just block and wait on ReentrantTransactionDispatcher.waitForPermits().

I could probably use two different environments, which would probably help by 1/2.... but it just feels like something is quite wrong, if I can't have two threads writing unrelated keys to xodus at the same time, without them contending.

As a side note, is this the best place to talk about xodus usage? I didn't see a forum or anything.

Upvotes: 0

Views: 335

Answers (1)

Vyacheslav Lukianov
Vyacheslav Lukianov

Reputation: 2053

Concurrent writes to a single Environment always contend even if written data seems to be completely decoupled (unrelated). There are two ways to reduce the contention:

  1. Put writes in order using the executeInExclusiveTransaction method instead of executeInTransaction. This approach will arrange not only user transactions, but background GC transactions as well. The disadvantage is that it orders also a preparatory work for writes (defined in a lambda passed to the method) which probably could be executed in parallel.
  2. Split single Environment onto several ones. If data written concurrently is really unrelated (i.e. at least it doesn't require some consistency), it can be saved in different transactions to different Environments. Log shared cache balances well with hundreds of Environments created in a single JVM unless you define exodus.log.cache.shared=false. In addition, different Environments can be physically isolated on different storage devices.

Upvotes: 1

Related Questions