CODEWITHSUNDEEP
activemq-classicleveldb
srini
srini

Reputation: 83

Anyone used LevelDB store for ActiveMQ message persistence?

At present we are using KahaDB store for message persistence in ActiveMQ and so far good.
As per the release notes of ActiveMQ5.6, LevelDB provides enhanced performance.
Has anyone tried usign LevelDB and if so could you provide the pros and cons?

Upvotes: 6

Views: 5550

Answers (5)

Roman Kharkovski
Roman Kharkovski

Reputation: 85

I did extensive testing of AMQ performance and was not able to gain any statistically significant difference between LevelDB vs. KahaDB in my tests: http://whywebsphere.com/2015/03/12/ibm-mq-vs-apache-activemq-performance-comparison-update/

Upvotes: 1

hyc
hyc

Reputation: 1443

Most of the performance claims made for LevelDB appear to be empty claims. It is supposed to support high concurrency reads but multi-threaded testing shows no concurrency gains. https://github.com/ayende/raven.voron/pull/9#issuecomment-29764803

(In contrast, LMDB shows perfect linear performance gains for reads across multiple CPUs. https://github.com/ayende/raven.voron/pull/9#issuecomment-29780359 )

Upvotes: 2

voodooless
voodooless

Reputation: 51

We've been using the levelDB store a month of two now in production on NFS (with standard file lock failover configured). We've had a corrupt store several times now in the last few weeks, with no errors in the logs... just queues piling up, and very low throughput. The only thing we could do to resolve this, is throw away the store, and start over.

So we've switched back to the old and reliable KahaDB store again for now.

Upvotes: 5

Hiram Chirino
Hiram Chirino

Reputation: 4091

FYI: Here's a link to the official docs for the ActiveMQ LevelDB Store

Cons:

  • It's a brand new store, so may still have some bugs left in it.
  • LevelDB indexes need to 'compact' occasionally which MIGHT stall out new writes.
  • You can't just delete the index and rebuild it from the data files like you can with KahaDB
  • KahaDB handles disk corruption much more gracefully, recovering what it can and discarding corrupted records.

Pros:

  • Append mostly disk access patterns improve perf on rotational disk.
  • Fewer disk syncs than KahaDB
  • Fewer index entries need to be inserted per message stored
  • Fewer index lookups needed to load a message from disk into memory
  • Uses Snappy compression to reduce the on disk size of index entries
  • Optional Snappy compression of data logs.
  • A send to a composite destination only stores the message on disk once.
  • Faster and more frequent data file GCs.
  • Has a 'Replicated' variation where it can self replicate to 'slave' brokers to ensure message level HA.

Upvotes: 6

inglor
inglor

Reputation: 337

Please see this link: https://github.com/fusesource/fuse-extra/tree/master/fusemq-leveldb#how-to-use-with-activemq-56 There's a small comparison for leveldb vs kahadb.

I am currently trying it out on a system with high message throughput , and I see better results already. I still need to see if it is stable, but so far good.

Upvotes: 3

Related Questions