Redis slow log shows issue?

Question

I pretty new to Redis, so I apologize in advance if this question is too basic.

I am using Redis in 2 places in my application.

1. I use two redis keys, and only incr them, but this happen many times and very often, about 10-20 times a second.
2. In a completely different area, I use more complex sets and hashs. The implementation requires for thousands of items, but not very frequent, a few times a day, and latency here is less important.

This is the slow log after 1 day:

The info shows that I have a lot of connections

redis_version:2.4.10
redis_git_sha1:00000000
redis_git_dirty:0
arch_bits:64
multiplexing_api:epoll
gcc_version:4.4.6
process_id:1769
uptime_in_seconds:190693
uptime_in_days:2
lru_clock:1725649
used_cpu_sys:386.48
used_cpu_user:200.63
used_cpu_sys_children:3.19
used_cpu_user_children:4.76
connected_clients:12
connected_slaves:0
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0
used_memory:6551904
used_memory_human:6.25M
used_memory_rss:22675456
used_memory_peak:7991472
used_memory_peak_human:7.62M
mem_fragmentation_ratio:3.46
mem_allocator:jemalloc-2.2.5
loading:0
aof_enabled:0
changes_since_last_save:62
bgsave_in_progress:0
last_save_time:1464291307
bgrewriteaof_in_progress:0
total_connections_received:222528
total_commands_processed:2635087
expired_keys:29
evicted_keys:0
keyspace_hits:12056
keyspace_misses:1465
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:1309
vm_enabled:0
role:master
db0:keys=64,expires=2

I have checked the code, and I don't see a reason why I have so many connections. The majority of usage is done from a single file that instantiates redis connection only once. This file is called maybe a few thousands times a day.

I am trying to understand should I be worried about this, and if there is something that I should do differently in the implementation.

1. If I put the two parts in different redis DB's, will it make any difference?
2. From what I understand in the slowlog, the latency in the incr command happened during the HMSET (and the INFO commands). Assuming that I need to deal with sets of >10K items, is there any way to avoid or minimize the latency on the critical INCR commands?
3. Does these slow commands has anything to do with the large number of connections?

I am afraid to further use redis in more parts of the application in case I hurt the performance of what is already some what working.

Any suggestions what should I check to analyze this further would be more than welcome

Answer 1

Few things to clarify before answering.

• Redis is single threaded. Even if you hit parallel requests it will be processed one by one. Others will wait.
• INCR command is not heavy, it will not come in slowlogs unless until it is waiting to be executed or blocking.
• Hmset is definitely heavy with >10 K items, this will be blocking.

Answers to your questions:

1. If I put the two parts in different redis DB's, will it make any difference?

No It won't. It has nothing to do with the load in your scenario.

2. From what I understand in the slowlog, the latency in the incr command happened during the HMSET (and the INFO commands). Assuming that I need to deal with sets of >10K items, is there any way to avoid or minimize the latency on the critical INCR commands?

latency depends on the load in the system. Critical ones are the ones that happens during the load. You can't have the control over there.

3. Does these slow commands has anything to do with the large number of connections?

Number of live connection alone matters. There is a difference between 10 live connection and 100 live connections each performing simultaneously.

Solution: HMSET commands are the bottlenecks here, because they are blocking. Instead of using HMSET for >10K elements split them into multiples of 100s or thousands. Try with different numbers and find an optimal one and fix that number. Also upgrade to higher versions of redis which will perform better than the previous ones.

p.s: Info command takes 1750 seconds in your screen shot which seems very unusual.