CODEWITHSUNDEEP
unixprocessredis
Thinker
Thinker

Reputation: 537

How Redis RDB persistance actually works behind the scene?

i was going through Redis RDB persistence. I having some doubts regarding RDB persistence related to its disadvantage.

Understanding So far:

We should use rdb persistence when we need to save the snapshot of dataset currently in memory at some regular interval.

I can understand that in this way we can lose some data in case of server break down. But another disadvantage that i can't understand is how fork can be time consuming when persisting large dataset using rdb.

Quoting from Documentation

RDB needs to fork() often in order to persist on disk using a child process. Fork() can be time consuming if the dataset is big, and may result in Redis to stop serving clients for some millisecond or even for one second if the dataset is very big and the CPU performance not great. AOF also needs to fork() but you can tune how often you want to rewrite your logs without any trade-off on durability.

I know how fork works as per my knowledge When parent process forks it create a new Child process and we can allow some code that child process will execute based on its pid or we can provide it some new executable that it will work on using exec() system call.

but things that i don't understand how it will be heavy task when size of dataset is larger?

I think i know the answer but i m not sure about that

Quoted from this link https://www.bottomupcs.com/fork_and_exec.xhtml

When a process calls fork then

the operating system will create a new process that is exactly the same as the parent process. This means all the state that was talked about previously is copied, including open files, register state and all memory allocations, which includes the program code.

As per above statement whole dataset of redis will be copied to child.

Am i understanding right?

Upvotes: 0

Views: 908

Answers (1)

nnog
nnog

Reputation: 1682

When standard fork is called with copy-on-write the OS must still copy all the page table entries, which can take time time if you have small 4k pages and a huge dataset, this is what makes the actual fork() time slow.

You can also find a lot of time and memory is required if your dataset is changing a lot in a sparse way, as copy-on-write semantics triggers the actual memory pages to be copied as changes are made to the original. Redis also performs incremental rehashing and maintains expiry etc. so an instance that is more active will typically take longer to save to disk.

More reading:

Faster forking of large processes on Linux?

http://kirkwylie.blogspot.co.uk/2008/11/linux-fork-performance-redux-large.html

Upvotes: 2

Related Questions