CODEWITHSUNDEEP
performanceloggingtransactionsberkeley-db
Pupkov-Zadnij
Pupkov-Zadnij

Reputation: 1392

How to use Berkeley Db transactions without logging/writing to __db.00x files?

I am developping an application that uses Berkeley Db. It must conform with ACI (but not ACID), and performance is very important. More than that, performace was the reason why I had chosen Berkeley Db. But now Berkeley Db seems to be a bottleneck because of logging. The problem is that the more data is stored in the database, the longer are the values to be stored. I make

    envp->set_flags(envp, DB_TXN_NOSYNC, 1);

And then open the environment with the flags:

    DB_CREATE     |  // Create the environment if it does not exist
    DB_INIT_TXN   |  // Initialize the transactional subsystem.
    DB_INIT_MPOOL |  // Initialize the memory pool (in-memory cache)
    DB_INIT_LOCK  |  // Initialize the locking subsystem
    DB_THREAD;

But it is still logging to the disk, and performace gets extremely poor, as the data amount increases and the values being stored get longer, I suppose, the great majority of time is spent on saving them into log-files -- load average is 

load average: 19.65, 19.12, 17.00

I don't need any logs syncing at runtime -- I need only ACI, but not ACID.

Is there a way to turn logging off or to disable writing logs to disk?

Edit: As nobody answered, I have googled a lot and found a way to turn off logging, but, unfortunately, it didn't help -- environment still writes a lot to __db.00x files, and I don't know why and how to make it in-memory.

Upvotes: 1

Views: 935

Answers (2)

Jeff Johnson
Jeff Johnson

Reputation: 2390

Disabling fsync(2) is one way to improve performance.

Write a stub routine that returns 0 and set the fsync vector when opening a Berkeley DB.

Upvotes: 0

hflzh
hflzh

Reputation: 644

Have you tried to configure in-memory logging? Please refer to the doc you mentioned http://docs.oracle.com/cd/E17076_02/html/api_reference/C/envlog_set_config.html

envp->set_flags(envp, DB_LOG_IN_MEMORY, 1);

Upvotes: 0

Related Questions