CODEWITHSUNDEEP
redis
jinreal
jinreal

Reputation: 1555

Memory allocation in redis

After browsing some of the code of Redis, I notice that the "robj" creation is done by calling "createObject" function, which in turn calls "malloc" to alloc memory. So whenever a request comes(e.g. SET uid 1234), there will be a memory allocation process.

My question is, why not pre-allocate the space to reduce memory allocation cost? because of different object size?

Upvotes: 2

Views: 4446

Answers (1)

Sripathi Krishnan
Sripathi Krishnan

Reputation: 31528

My question is, why not pre-allocate the space to reduce memory allocation cost? because of different object size?

What you describe is the job of a memory allocator. Building that kind of logic within Redis isn't trivial.

By default, Redis uses jemalloc memory allocator on Linux.

Jemalloc has several optimizations to reduce memory allocation overheads. Read the following links to learn how jemalloc helps -

  1. Facebook Engineering blog on Jemalloc
  2. Jemalloc Paper from 2006

When Redis introduced Jemalloc, lots of memory management problems just disappeared. Here is what antirez (author of redis) had to say about introducing jemalloc:

[snip] An allocator is a serious thing. Since we introduced the specially encoded data types Redis started suffering from fragmentation. We tried different things to fix the problem, but basically the Linux default allocator in glibc sucks really, really hard.

Including jemalloc inside of Redis was a huge win. Every single case of fragmentation in real world systems was fixed by this change, and also the amount of memory used dropped a bit.

P.S. You are likely going to get better and more informed answers on the redis-db mailing list

Upvotes: 8

Related Questions