How to configure RabbitMQ Cluster on Windows

Question

Please, I need your kind help.

I am struggling since last week trying to configure RabbitMQ cluster on windows servers. I reinstalled RabbitMQ 3.7.8 and ErlangOTP 21-1 for more than 15 times trying to figure out the issue and fix it, but unfortunately, all of my attempts failed.

My environment: Three virtual machines with OS: windows server 2012

Environment Variable setting:

•   ERLANG_HOME = C:\Program Files\erl10.1
•   path: C:\Program Files\Handle
•   %RABBITMQ_BASE% = c:\RabbitMQ
•   %RABBITMQ_CONFIG_FILE% = %RABBITMQ_BASE%
•   RABBITMQ_NODENAME = masternode
•   Server names: RabbitMQ01 “masternode” , RabbitMQ02, RabbitMQ03
•   Adding IP & hots name “masternode” in the host file for name resolution (C:\Windows\System32\drivers\etc\hosts)
•   Setting RabbitMQ configuration file “advanced.config” in c:\users\USERNAME\AppData\Roaming\RabbitMQ” to enable users to login to RabbitMQ management remotely by adding the following setting
[
{rabbit,
  [
   {tcp_listeners, [5672]},
   {loopback_users, []},

   {num_tcp_acceptors, 100}
  ]
  }
].

Installing steps: (for each server)

1. After setting the environment variables, I restarted the machine. Then
2. Install ErlangOTP 21-1
3. Install RabbitMQ 3.7.8 server
4. Enable plugins - rabbitmq-plugins.bat enable rabbitmq_management 

After completing the installation of the above servers, I start configuring the cluster

1. First, I verified that masternode is reachable from RabbitMQ02, 02 by pinging Rabbit@masternode and it’s successful.

2. Stopped RabbitMQ server and App by using the following command

   rabbitmq-service stop rabbitmqctl stop_app

   3. Copy .erlang.cookie from masternode “path=c:\windows\system32\config\ systemprofile” to RabbitMQ02 “path= c:\windows\system32\config\ systemprofile” and “path=c:\users\USERNAME”.

3. Start RabbitMQ service & App by executing the below commands

rabbitmqctl start_app rabbitmq-service start

5. Stop App again by executing this command “rabbitmqctl stop_app”
6. I start joining RabbitMQ02 to masternode rabbitmqctl join_cluster rabbit@masternode (small case)

Once I reach this point, I’m getting the following error on RabbitMQ02.

C:\Program Files\RabbitMQ Server\rabbitmq_server-3.7.8\sbin>rabbitmqctl join_cluster rabbit@masternode Error: unable to perform an operation on node 'rabbit@rhrabbitmq02'. Please see diagnostics information and suggestions below.

Most common reasons for this are:

• Target node is unreachable (e.g. due to hostname resolution, TCP connection o r firewall issues)
• CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server)
• Target node is not running

In addition to the diagnostics info below:

• See the CLI, clustering and networking guides on http://rabbitmq.com/document ation.html to learn more
• Consult server logs on node rabbit@rhrabbitmq02

DIAGNOSTICS

attempted to contact: [rabbit@rhrabbitmq02]

rabbit@rhrabbitmq02: * connected to epmd (port 4369) on rhrabbitmq02 * epmd reports node 'rabbit' uses port 25672 for inter-node and CLI tool traff ic * TCP connection succeeded but Erlang distribution failed

• Authentication failed (rejected by the remote node), please check the Erlang cookie

Current node details: * node name: rabbitmqcli41@rhrabbitmq02 * effective user's home directory: C:\Users\admin * Erlang cookie hash: pndBksvQokXyiSqBdnne/g==

masternode Status:

C:\Program Files\RabbitMQ Server\rabbitmq_server-3.7.8\sbin>rabbitmqctl status
Status of node masternode@rhrabbitmq01 ...
[{pid,4008},
 {running_applications,
     [{rabbitmq_management,"RabbitMQ Management Console","3.7.8"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.7.8"},
      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.7.8"},
      {rabbit,"RabbitMQ","3.7.8"},
      {amqp_client,"RabbitMQ AMQP Client","3.7.8"},
      {rabbit_common,
          "Modules shared by rabbitmq-server and rabbitmq-erlang-client",
          "3.7.8"},
      {os_mon,"CPO  CXC 138 46","2.4.6"},
      {cowboy,"Small, fast, modern HTTP server.","2.2.2"},
      {mnesia,"MNESIA  CXC 138 12","4.15.5"},
      {cowlib,"Support library for manipulating Web protocols.","2.1.0"},
      {lager,"Erlang logging framework","3.6.3"},
      {syslog,"An RFC 3164 and RFC 5424 compliant logging framework.","3.4.3"},
      {ranch_proxy_protocol,"Ranch Proxy Protocol Transport","1.5.0"},
      {jsx,"a streaming, evented json parsing toolkit","2.8.2"},
      {ranch,"Socket acceptor pool for TCP protocols.","1.5.0"},
      {ssl,"Erlang/OTP SSL application","9.0.2"},
      {public_key,"Public key infrastructure","1.6.2"},
      {asn1,"The Erlang ASN1 compiler version 5.0.7","5.0.7"},
      {inets,"INETS  CXC 138 49","7.0.2"},
      {xmerl,"XML parser","1.3.18"},
      {recon,"Diagnostic tools for production use","2.3.2"},
      {crypto,"CRYPTO","4.3.3"},
      {goldrush,"Erlang event stream processor","0.1.9"},
      {compiler,"ERTS  CXC 138 10","7.2.5"},
      {syntax_tools,"Syntax tools","2.1.6"},
      {sasl,"SASL  CXC 138 11","3.2.1"},
      {stdlib,"ERTS  CXC 138 10","3.6"},
      {kernel,"ERTS  CXC 138 10","6.1"}]},
 {os,{win32,nt}},
 {erlang_version,
     "Erlang/OTP 21 [erts-10.1] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:64
]\n"},
 {memory,
     [{connection_readers,0},
      {connection_writers,0},
      {connection_channels,0},
      {connection_other,29212},
      {queue_procs,0},
      {queue_slave_procs,0},
      {plugins,1192744},
      {other_proc,26188552},
      {metrics,195524},
      {mgmt_db,337648},
      {mnesia,75088},
      {other_ets,2748464},
      {binary,181696},
      {msg_index,27840},
      {code,27573597},
      {atom,1131721},
      {other_system,9846202},
      {allocated_unused,9377056},
      {reserved_unallocated,0},
      {strategy,rss},
      {total,[{erlang,69528288},{rss,78905344},{allocated,78905344}]}]},
 {alarms,[]},
 {listeners,
     [{clustering,25672,"::"},
      {amqp,5672,"::"},
      {amqp,5672,"0.0.0.0"},
      {http,15672,"::"},
      {http,15672,"0.0.0.0"}]},
 {vm_memory_calculation_strategy,rss},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,3435785420},
 {disk_free_limit,50000000},
 {disk_free,86038073344},
 {file_descriptors,
     [{total_limit,8092},
      {total_used,2},
      {sockets_limit,7280},
      {sockets_used,0}]},
 {processes,[{limit,1048576},{used,399}]},
 {run_queue,1},
 {uptime,6087},
 {kernel,{net_ticktime,60}}]

Answer 1

Problem solution: setting RABBITMQ_NODENAME incorrectly “masternode” instead of “rabbit@masternode”, Error in command execution sequence which crucial thing to perform (when joining the cluster - stop_app first then service stop), and restarting the service after changing the configuration instead of reinstalling the rabbitmq service.

I will be happy to share all of the steps details if needed for installing, configuring rabbitmq, and configuring the cluster on windows based on my setup scenario. Best regards.