Redis on Ubuntu 14.04

Question

I installed redis 2.8.12 from ppa but fails to start. Looking at the log this is what I get:

[4886 | signal handler] (1405388991) Received SIGTERM, scheduling shutdown...
[4886] 14 Jul 20:49:51.561 # User requested shutdown...
[4886] 14 Jul 20:49:51.561 * Saving the final RDB snapshot before exiting.
[4886] 14 Jul 20:49:51.566 * DB saved on disk
[4886] 14 Jul 20:49:51.566 * Removing the pid file.
[4886] 14 Jul 20:49:51.566 # Redis is now ready to exit, bye bye...
[6726] 14 Jul 20:56:04.063 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
[6726] 14 Jul 20:56:04.063 # Redis can't set maximum open files to 10032 because of OS error: Operation not permitted.
[6726] 14 Jul 20:56:04.063 # Current maximum open files is 1024. maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 2.8.12 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in stand alone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 6726
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

[6726] 14 Jul 20:56:04.064 # Server started, Redis version 2.8.12
[6726] 14 Jul 20:56:04.064 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
[6726] 14 Jul 20:56:04.064 * The server is now ready to accept connections on port 6379

I did ran the command sysctl vm.overcommit_memory=1 and rebooted as suggested but did not work. I'm not exactly what to do a this point. Does anybody has an idea?

Answer 1

Simply do

sudo echo 1 > /proc/sys/vm/overcommit_memory

It will solve you that error. :)

Background saving is failing with a fork() error under Linux even if I've a lot of free RAM!

Short answer: echo 1 > /proc/sys/vm/overcommit_memory :)

And now the long one:

Redis background saving schema relies on the copy-on-write semantic of fork in modern operating systems: Redis forks (creates a child process) that is an exact copy of the parent. The child process dumps the DB on disk and finally exits. In theory the child should use as much memory as the parent being a copy, but actually thanks to the copy-on-write semantic implemented by most modern operating systems the parent and child process will share the common memory pages. A page will be duplicated only when it changes in the child or in the parent. Since in theory all the pages may change while the child process is saving, Linux can't tell in advance how much memory the child will take, so if the overcommit_memory setting is set to zero fork will fail unless there is as much free RAM as required to really duplicate all the parent memory pages, with the result that if you have a Redis dataset of 3 GB and just 2 GB of free memory it will fail. Setting overcommit_memory to 1 says Linux to relax and perform the fork in a more optimistic allocation fashion, and this is indeed what you want for Redis. A good source to understand how Linux Virtual Memory work and other alternatives for overcommit_memory and overcommit_ratio is this classic from Red Hat Magazine, "Understanding Virtual Memory". Beware, this article had 1 and 2 configuration values for overcommit_memory reversed: refer to the proc(5) man page for the right meaning of the available values.

Source: http://redis.io/topics/faq

Answer 2

When you get the overcommit warning, the server is up and running. However, when it will fork another process to save the database (RDB) or compact the AOF, you may get an error if the memory does not account for the copy-on-write overhead.

This is why Redis insists on having the overcommit mode set to 1.

Now, when you use sysctl to change the kernel parameters, they are only changed at runtime. Since the machine has been rebooted, you have just lost this change. If you want the definition to be permanent, it needs also to be added in file /etc/sysctl.conf

Note that on top of the overcommit memory issue, you also have an issue regarding the number of file descriptors. This can be increased by tweaking the /etc/security/limits.conf file (nofile parameter), or in some conditions, the ulimit settings of your session.