CODEWITHSUNDEEP
redisredis-sentinel
kmarabet
kmarabet

Reputation: 1117

In Redis-sentinel master-slave cofiguration new master is shown as down for the initial master

Have installed a redis (v. 3.0.4) master-slave model using 3 nodes (1 master and 2 slaves) with requirepass for each node as described in https://www.digitalocean.com/community/tutorials/how-to-configure-a-redis-cluster-on-ubuntu-14-04 then started 3 sentinel on each node as described in the article http://blog.commando.io/redis-is-easy-trivial-hard/

After have tried to take down the master, sentinel has promoted one of the slaves to a master as expected. Then when the old master was up again it became a slave and recognize a new master, this could be seen in the /etc/redis/sentinel.conf which was updated with the new master IP in 'sentinel monitor redis-cluster' attribute. But have noticed that the old master despite knowing the new master IP, it considers the new master as down, unlike the other slave which see it up. This could checked by running this command against the old master: 

$redis-cli -a altoros info replication
#
Replication
role:slave
master_host: new master ip
master_port:6379
master_link_status:down

This also seems to be causing the following error "MASTERDOWN Link with MASTER is down and slave-serve-stale-data is set to 'no'", when trying to use a synchronous client for testing data replication over nodes.

The logs of the old masters (/var/log/redis/redis-server.log) are showing:
20731:S 09 Nov 10:16:31.117 * Connecting to MASTER <new master="" ip="">: 6379
20731:S 09 Nov 10:16:31.117 * MASTER <-> SLAVE sync started
20731:S 09 Nov 10:16:31.118 * Non blocking connect for SYNC fired the event. 
20731:S 09 Nov 10:16:31.118 * Master replied to PING, replication can continue...
20731:S 09 Nov 10:16:31.119 * (Non critical) Master does not under stand REPLCONF listening-port: -NOAUTH Authentication required.
20731:S 09 Nov 10:16:31.119 * (Non critical) Master does not under stand REPLCONF capa:
-NOAUTH Authentication required.

This looks like the old master cannot authenticate to the new master, because it doesn't have his password, but how to set that properly?

Because have noticed that /etc/redis/redis.conf did not changed after a new master was promoted, unlike /etc/redis/sentinel.conf, and this could cause that as redis.conf of the master doesn't the password of the new master.

Would appreciate any hint to resolve the issue, thanks in advance.

Upvotes: 1

Views: 3629

Answers (1)

The Real Bill
The Real Bill

Reputation: 15813

The master needs to be configured just like a slave because it might become one, someday. As such you need to set it's masterauth to the password for the pod.

You can do this without restarting y doing the following against the "old master":

redis-cli -h oldmasterip -a thepassword config set masterauth thepassword
redis-cli -h oldmasterip -a thepassword config rewrite

And it should be fine from that point, and the config file will be updated.

Upvotes: 4

Related Questions