MySQL Zombie Dump threads

Question

I'm getting the below constantly logged on my master mysql server. I've googled this to death but with no luck.

Can anyone give me any advice.

2016-07-21T08:17:43.765005Z 41323 [Note] Start binlog_dump to master_thread_id(41323) slave_server(4), pos(, 4)
2016-07-21T08:17:51.598518Z 41324 [Note] While initializing dump thread for slave with UUID <5dbbd9c5-2963-11e5-9b69-1458d042a2f8>, found a zombie dump thread with the same UUID. Master is killing the zombie dump thread.
2016-07-21T08:17:51.615993Z 41324 [Note] Start binlog_dump to master_thread_id(41324) slave_server(3), pos(, 4)
2016-07-21T08:18:12.578065Z 41326 [Note] While initializing dump thread for slave with UUID <60f57c3e-2963-11e5-9b69-1458d057f760>, found a zombie dump thread with the same UUID. Master is killing the zombie dump thread.
2016-07-21T08:18:12.656642Z 41326 [Note] Start binlog_dump to master_thread_id(41326) slave_server(1), pos(, 4)

Answer 1

Try to raise the slave_net_timeout. Prior to version 5.7.7, it was set to 3600, now to 60. Therefore, if you have any trouble on the network, the slave will think it has lost connection the server but when a Slave tries to [re]connect, it creates a dump thread on Master. Before Master creates a dump thread, it checks if there is already a dump thread running for that Slave (using the UUID value). If yes, it considers that old dump thread as "Zombie" dump thread and kills it. Creates a new one and prints this "Note" in the error log file. (This is cited from the official MySQL site) Practically speaking: Enter MySQL on the slave machine and

mysql> set global slave_net_timeout=3600;

Restart your MySQL server and it should solve the problem.

Once this is OK, you can add this parameter in the configuration file my.conf to make the changes definitive.

Answer 2

This is a bug in MySQL 5.6+, fixed in next release. As explained here, the master was trying to identify different slaves based on their server_uuid, which was empty pre-5.6