CODEWITHSUNDEEP
rocksdbrocksdb-java
user9959339
user9959339

Reputation:

can I backing up rocksdb while putting?

I want to back up rocksdb while not blocking my writing calls, the status I expected is rocksdb should back up the data at that moment and ignoring the writing calls while backing up, is there a proper way to accomplish this?

I tested this in java using rocksdb-jni:

    RocksDB.loadLibrary();
    RocksDB rocksDb = RocksDB.open("C:/rocksdb/data");
    BackupEngine engine = BackupEngine.open(rocksDb.getEnv(), new BackupEngineOptions("C:/rocksdb/backup"));
    for(int i = 0;i < 100000;i++){
        byte[] k = ByteBuffer.allocate(4).putInt(0, i).array();
        rocksDb.put(k,k);
    }
    ExecutorService executorService = Executors.newFixedThreadPool(2);
    CountDownLatch countDownLatch = new CountDownLatch(1);
    executorService.execute(() -> {
        try {
            Thread.currentThread().setPriority(Thread.MAX_PRIORITY);
            countDownLatch.await();
            engine.createNewBackup(rocksDb, false);
        } catch (RocksDBException | InterruptedException e) {
            throw new RuntimeException(e);
        }
    });
    executorService.execute(() -> {
        try {
            Thread.currentThread().setPriority(Thread.NORM_PRIORITY);
            countDownLatch.await();
            //Thread.sleep(1);
            for(int i = 100000;i < 200000;i++){
                byte[] k = ByteBuffer.allocate(4).putInt(0, i).array();
                rocksDb.put(k,k);
            }
        } catch (InterruptedException | RocksDBException e) {
            throw new RuntimeException(e);
        }
    });
    countDownLatch.countDown();

the result I expected from restoring is exactly 100000 records, but I always got more records

Upvotes: 0

Views: 441

Answers (1)

Asad Awadia
Asad Awadia

Reputation: 1521

Yes - both backup and checkpoint engines can achieve this i.e. take a live backup while not blocking writes

https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB

Upvotes: 1

Related Questions