CODEWITHSUNDEEP
lmdb
ivenhov
ivenhov

Reputation: 321

Is there a limit of sub-databases in LMDB?

Posting here as I could not find any forums for lmdb key-value store.

Is there a limit for sub-databases? What is a reasonable number of sub-databases concurrently open?

I would like to have ~200 databases which seems like a lot and clearly indicates my model is wrong. I suppose could remodel and embed id of each db in key itself and keep one db only but then I have longer keys and also I cannot drop database if needed.

I'm interested though if LMDB uses some sort of internal prefixes for keys already. Any suggestions how to address that problem appreciated.

Upvotes: 7

Views: 3005

Answers (2)

rouzier
rouzier

Reputation: 1180

If you read the documentation for mdb_env_set_maxdbs.

Currently a moderate number of slots are cheap but a huge number gets expensive: 7-120 words per transaction, and every mdb_dbi_open() does a linear search of the opened slots.

http://www.lmdb.tech/doc/group__mdb.html#gaa2fc2f1f37cb1115e733b62cab2fcdbc

The best way to know is to test the function call mdb_dbi_open performance to see if it is acceptable.

Upvotes: 4

Kanti Varanasi
Kanti Varanasi

Reputation: 89

Instead of calling mdb_dbi_open each time, keep your own map with database names to database handles returned from mdb_dbi_open. Reuse these handles for the lifetime of your program. This will allow you to have multiple databases within an environment and prevent the overhead with mdb_dbi_open.

Upvotes: 5

Related Questions