CODEWITHSUNDEEP
ruby-on-railsrubynetwork-programmingredis
netflux
netflux

Reputation: 3838

Redis: weird protocol/network errors

I'm running Redis and connecting from Ruby using ezmobius's Redis gem[1].

Periodically (about once a day) I get a series of exceptions in my Rails app caused by Redis returning strange results.

They are often triggered by an exception such at this:

Redis::ProtocolError: Protocol error, got '3' as initial reply byte

or

Redis::ProtocolError: Protocol error, got '9' as initial reply byte

or sometimes

Errno::EAGAIN: Resource temporarily unavailable - Timeout reading from the socket

It usually requires a restart of my Rails servers to clear up the connection problem. I'm running Fedora Core 8, Rails 2.3.8, Redis gem 2.0.3. I've got the system_timer gem installed. Anybody got any ideas how I can stop these errors?

[1]Redis gem

Upvotes: 1

Views: 3227

Answers (4)

Ivan Tishchenko
Ivan Tishchenko

Reputation: 106

Resource temporarily unavailable - Timeout reading from the socket

I had an identical issue, but with the C++ version of the Redis client. I was trying to reuse one connection from 16 different threads while fetching ~100 keys.

In my case it helped to increase the parameters socket_timeout and connect_timeout.

    sw::redis::ConnectionOptions opt;
    opt.host = host;
    opt.port = port;
    opt.socket_timeout = REDIS_SOCKET_TIMEOUT;
    opt.connect_timeout = REDIS_CONNECTION_TIMEOUT;

https://github.com/sewenew/redis-plus-plus/issues/604

The Ruby version of the Redis client library should also allow to control the timeouts.

Upvotes: 0

George
George

Reputation: 4473

I had a slightly similar issue with

Errno::EAGAIN: Resource temporarily unavailable - Timeout reading from the socket

Turns out, my redis-server had a timeout set to 300 seconds on connections. After 5 minutes, redis was killing the connection to my workers and they were logging the error above.

If your socket timeouts are happening every x seconds, its no doubt redis killing your 'idle' connection!

Upvotes: 2

djanowski
djanowski

Reputation: 5858

When initializing the connection, make sure to pass the :thread_safe option:

Redis.connect(:thread_safe => true)

Upvotes: 0

Kuba Suder
Kuba Suder

Reputation: 7827

I've just noticed the same thing in my background workers, which store tasks in queues in Redis and also communicate through Redis pub/sub. Google results suggest that this can happen if you use the same Redis object from more than one thread... I'm not sure if this is the case in my app, I'll have to investigate this (but I do have threads there).

Upvotes: 1

Related Questions