CODEWITHSUNDEEP
javastoragekey-valuedisk-based
Andrey Logvinov
Andrey Logvinov

Reputation: 119

Java disk-based key-value storage

Is there an efficient Java implementation of a filesystem-based key-value storage with the following features:

  1. Store, overwrite, and retrieve byte arrays by a unique ID (may be assigned by the storage)
  2. No memory caching (read means read from file system, write means write to file system immediately)
  3. Total data size up to few terabytes
  4. Number of stored objects up to hundreds of millions
  5. Manageable number of file system objects (to move/copy/delete entire storage on file system level)

Will Berkeley DB JE do?

Upvotes: 10

Views: 4302

Answers (4)

Zhou Tai
Zhou Tai

Reputation: 214

I suggest MapDB, MapDB provides concurrent Maps, Sets and Queues backed by disk storage or off-heap memory. Lightweight and hackable.

Upvotes: 2

meriton
meriton

Reputation: 70564

Simply format a dedicated partition with a file system of your choice? The file system would meet requirements 1-4, and requirement 5 can be met by moving/copying or deleting that partition.

Upvotes: 3

user1098798
user1098798

Reputation: 381

Perhaps HBase, however you would need to run the whole Hadoop stack, which may well be overkill! http://hbase.apache.org/

Upvotes: 0

Stas
Stas

Reputation: 1725

This may work. Looks like your case. Suppose, worth having a look.

http://xtreemfs.blogspot.com/2008/11/babudb-efficient-key-value-store-for.html

Here is also presentation about that thing with details how it works:

http://www.xtreemfs.org/slides/BabuDB-SNAPI.pdf

Upvotes: 0

Related Questions