CODEWITHSUNDEEP
hazelcast
dt94
dt94

Reputation: 117

Multiple Hazelcast instances on same machine?

I am trying to write unit tests for my Hazelcast cluster - According to this doc https://hazelcast.zendesk.com/hc/en-us/articles/115004425243-How-do-I-test-my-Hazelcast-cluster-

i wrote my code in this way - 

public void test(){
    HazelcastInstance instance = Hazelcast.newHazelcastInstance(); 
    HazelcastInstance instance2 = Hazelcast.newHazelcastInstance();                
}

My console shows - 

Members [1] {
    Member [MY IP]:5701 -  this
}

and

Members [1] {
    Member [MY IP]:5702 - this
}

The port number changes, IP stays the same but they are not in the same cluster. What do I do to get them to be a part of the same cluster (on a single JVM)? UPDATE

More console data -

Jul 31, 2018 3:21:00 PM com.hazelcast.config.XmlConfigLocator
INFO: Loading 'hazelcast-default.xml' from classpath.
Jul 31, 2018 3:21:01 PM com.hazelcast.instance.DefaultAddressPicker
INFO: [LOCAL] [dev] [3.7.4] Prefer IPv4 stack is true.
Jul 31, 2018 3:21:01 PM com.hazelcast.instance.DefaultAddressPicker
INFO: [LOCAL] [dev] [3.7.4] Picked [MY_IP]:5701, using socket     ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
Jul 31, 2018 3:21:01 PM com.hazelcast.system
INFO: [MY_IP]:5701 [dev] [3.7.4] Hazelcast 3.7.4 (20161209 - 3df1bb5) starting at [MY_IP]:5701
Jul 31, 2018 3:21:01 PM com.hazelcast.system
INFO: [MY_IP]:5701 [dev] [3.7.4] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
Jul 31, 2018 3:21:01 PM com.hazelcast.system
INFO: [MY_IP]:5701 [dev] [3.7.4] Configured Hazelcast Serialization version : 1
Jul 31, 2018 3:21:01 PM com.hazelcast.spi.impl.operationservice.impl.BackpressureRegulator
INFO: [MY_IP]:5701 [dev] [3.7.4] Backpressure is disabled
Jul 31, 2018 3:21:01 PM com.hazelcast.instance.Node
INFO: [MY_IP]:5701 [dev] [3.7.4] Creating MulticastJoiner
Jul 31, 2018 3:21:01 PM com.hazelcast.core.LifecycleService
INFO: [MY_IP]:5701 [dev] [3.7.4] [MY_IP]:5701 is STARTING
Jul 31, 2018 3:21:02 PM com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
INFO: [MY_IP]:5701 [dev] [3.7.4] Starting 4 partition threads
Jul 31, 2018 3:21:02 PM     com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
INFO: [MY_IP]:5701 [dev] [3.7.4] Starting 3 generic threads (1 dedicated for priority tasks)
Jul 31, 2018 3:21:02 PM com.hazelcast.nio.tcp.nonblocking.NonBlockingIOThreadingModel
INFO: [MY_IP]:5701 [dev] [3.7.4] TcpIpConnectionManager configured with Non Blocking IO-                    threading model: 3 input threads and 3 output threads
Jul 31, 2018 3:21:05 PM com.hazelcast.internal.cluster.impl.MulticastJoiner
INFO: [MY_IP]:5701 [dev] [3.7.4]



Members [1] {
    Member [MY_IP]:5701 - 
this
}

Jul 31, 2018 3:21:05 PM com.hazelcast.core.LifecycleService
INFO: [MY_IP]:5701 [dev] [3.7.4] [MY_IP]:5701 is 
STARTED
Jul 31, 2018 3:21:05 PM com.hazelcast.config.XmlConfigLocator
INFO: Loading 'hazelcast-default.xml' from classpath.
Jul 31, 2018 3:21:05 PM com.hazelcast.instance.DefaultAddressPicker
INFO: [LOCAL] [dev] [3.7.4] Prefer IPv4 stack is true.
Jul 31, 2018 3:21:05 PM com.hazelcast.instance.DefaultAddressPicker
INFO: [LOCAL] [dev] [3.7.4] Picked [MY_IP]:5702, using socket 
ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5702], bind any local is 
true
Jul 31, 2018 3:21:05 PM com.hazelcast.system
INFO: [MY_IP]:5702 [dev] [3.7.4] Hazelcast 3.7.4 (20161209 - 
3df1bb5) starting at [MY_IP]:5702
Jul 31, 2018 3:21:05 PM com.hazelcast.system
INFO: [MY_IP]:5702 [dev] [3.7.4] Copyright (c) 2008-2016, 
Hazelcast, Inc. All Rights Reserved.
Jul 31, 2018 3:21:05 PM com.hazelcast.system
INFO: [MY_IP]:5702 [dev] [3.7.4] Configured Hazelcast 
Serialization version : 1
Jul 31, 2018 3:21:05 PM 
com.hazelcast.spi.impl.operationservice.impl.BackpressureRegulator
INFO: [MY_IP]:5702 [dev] [3.7.4] Backpressure is disabled
Jul 31, 2018 3:21:05 PM com.hazelcast.instance.Node
INFO: [MY_IP]:5702 [dev] [3.7.4] Creating MulticastJoiner
Jul 31, 2018 3:21:05 PM com.hazelcast.core.LifecycleService
INFO: [MY_IP]:5702 [dev] [3.7.4] [MY_IP]:5702 is                     
Jul 31, 2018 3:21:05 PM 
com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
INFO: [MY_IP]:5702 [dev] [3.7.4] Starting 4 partition threads
Jul 31, 2018 3:21:05 PM 
com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl
INFO: [MY_IP]:5702 [dev] [3.7.4] Starting 3 generic threads (1 
dedicated for priority tasks)
Jul 31, 2018 3:21:05 PM 
com.hazelcast.nio.tcp.nonblocking.NonBlockingIOThreadingModel
INFO: [MY_IP]:5702 [dev] [3.7.4] TcpIpConnectionManager 
configured with Non Blocking IO-threading model: 3 input threads and 3 
output threads
Jul 31, 2018 3:21:08 PM 
com.hazelcast.internal.cluster.impl.MulticastJoiner
INFO: [MY_IP]:5702 [dev] [3.7.4] 


Members [1] {
    Member [MY_IP]:5702 - 
this
}

Jul 31, 2018 3:21:08 PM com.hazelcast.instance.Node
WARNING: [MY_IP]:5702 [dev] [3.7.4] Config seed port is 5701 
and cluster size is 1. Some of the ports seem occupied!
Jul 31, 2018 3:21:08 PM com.hazelcast.core.LifecycleService
INFO: [MY_IP]:5702 [dev] [3.7.4] [MY_IP]:5702 is 
STARTED

Upvotes: 1

Views: 7755

Answers (3)

MOL
MOL

Reputation: 31

after going through that stuff: https://docs.hazelcast.org/docs/1.9/manual/html/ch16s02.html

i've concluded that I just need a better config to tell Hazelcast to threat 2 instances as 1 cluster.

private val hazelcastOnNode1 = newHazelcastInstance("hazelcastInstance-cluster-node1")
private val hazelcastOnNode2 = newHazelcastInstance("hazelcastInstance-cluster-node2")

@BeforeAll
fun setupTest() {
    log.info("hazelcastOnNode1: ${hazelcastOnNode1.name} is part of a cluster: ${hazelcastOnNode1.cluster}")
    log.info("hazelcastOnNode2: ${hazelcastOnNode2.name} is part of a cluster: ${hazelcastOnNode2.cluster}")
}

private fun newHazelcastInstance(instanceName: String): HazelcastInstance {
    val cfg = Config()
    cfg.instanceName = instanceName
    cfg.groupConfig.name = "hazel-cluster"
    cfg.networkConfig.join.multicastConfig.isEnabled = false
    cfg.networkConfig.join.tcpIpConfig.isEnabled = true
    cfg.networkConfig.join.tcpIpConfig.members = listOf("127.0.0.1")
    return Hazelcast.newHazelcastInstance(cfg)
}

Upvotes: 3

Navid Mitchell
Navid Mitchell

Reputation: 1445

The default configuration will work fine for multiple instances within a single JVM or multiple JVM's on the same machine.

The error Config seed port is 5701 and cluster size is 1. Some of the ports seem occupied! is a Red Herring.

When you start multiple instances on the same machine it WILL pick a different port. However, the error above does indicate there is a communication problem between your instances.

When there is no communication problem you will see different output in the log. The secondary instance when started will output something like this (notice it is still using 5702).

com.hazelcast.core.LifecycleService      : [192.168.0.103]:5702 [dev] [3.11.1] [192.168.0.103]:5702 is STARTING
c.h.i.cluster.impl.MulticastJoiner       : [192.168.0.103]:5702 [dev] [3.11.1] Trying to join to discovered node: [192.168.0.103]:5701
com.hazelcast.nio.tcp.TcpIpConnector     : [192.168.0.103]:5702 [dev] [3.11.1] Connecting to /192.168.0.103:5701, timeout: 0, bind-any: true

In my case solving this was as simple as disabling the firewall on my local machine since I was binding to a DHCP address assigned by my router. If you are binding to the loopback address, you will want to ensure that the loopback on your OS works correctly with multicast. OSX has some known Java problems when using multicast with the loopback interface. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=7122846

Upvotes: 2

Neil Stevenson
Neil Stevenson

Reputation: 3150

As you noted, you're using the default Hazelcast configuration, as shown in the logs

INFO: Loading 'hazelcast-default.xml' from classpath.

The default Hazelcast configuration is for multicast. This is ideal for getting started quickly with multiple machines, not so good for one machine.

The first Hazelcast.newHazelcastInstance() will start a multicast group on port 5701.

The second Hazelcast.newHazelcastInstance() will find port 5701 in use, will take the next port 5702, and start a multicast group on 5702.

In other words, the second instance can be on the same port so can't be in the same group (as the group is based on the port). So you end up with two clusters which isn't what you want.

There are four steps to achieve what you want.

  1. You need a hazelcast.xml file in your test classpath.
  2. Turn off multicast
  3. Turn on another discovery, TCP is far and away the easiest here
  4. (Optional) Don't use 3.7.4, it's not the latest, go for 3.10.4

Turning off multicast does not turn on TCP. You could chose to have no discovery (useful for some tests).

So you need a hazelcast.xml that looks like this perhaps:

<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.10.xsd"
       xmlns="http://www.hazelcast.com/schema/config"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

  <network> 
    <join>  
      <multicast enabled="false"/>
      <tcp-ip enabled="true">
        <interface>127.0.0.1</interface>
      </tcp-ip>
    </join>
  </network>
</hazelcast>

Upvotes: 6

Related Questions