CODEWITHSUNDEEP
hazelcastvert.x
Mallikarjun
Mallikarjun

Reputation: 129

Can Hazelcast connect as a client to existing Hazelcast cluster instead of joining as a member of the cluster to implement vertx clustering

We are currently using vertx and hazelcast as its clustering implementation. For it to work as per the docs hazelcast is embedded inside our application meaning it will join as a member of the cluster. We would like our application to be independent of Hazelcast. The reason is when ever Hazelcast cache becomes inconsistent we are bringing down all our servers and restarting. Instead we would like to keep Hazelcast to its own server and connect vertx as a client so we restart hazelcast independent of our application server. Zookeeper cluster implementation does exactly how we would like but we don't want to maintain another cluster for just this purpose because we are also using Hazelcast for other cache purposes internal to our application. Currently we are doing some thing like this to make vertx work.

        Config hazelcastConfig = new Config();

        //Group
        GroupConfig groupConfig = new GroupConfig();
        groupConfig.setName(hzGroupName);
        groupConfig.setPassword(groupPassword);
        hazelcastConfig.setGroupConfig(groupConfig);

        //Properties
        Properties properties = new Properties();
        properties.setProperty("hazelcast.mancenter.enabled", "false");
        properties.setProperty("hazelcast.memcache.enabled", "false");
        properties.setProperty("hazelcast.rest.enabled", "false");
        properties.setProperty("hazelcast.wait.seconds.before.join", "0");
        properties.setProperty("hazelcast.logging.type", "jdk");
        hazelcastConfig.setProperties(properties);

        //Network
        NetworkConfig networkConfig = new NetworkConfig();
        networkConfig.setPort(networkPort);
        networkConfig.setPortAutoIncrement(networkPortAutoincrement);

        //Interfaces
        InterfacesConfig interfacesConfig = new InterfacesConfig();
        interfacesConfig.setEnabled(true);
        interfacesConfig.setInterfaces(interfaces);
        networkConfig.setInterfaces(interfacesConfig);

        //Join
        JoinConfig joinConfig = new JoinConfig();
        MulticastConfig multicastConfig = new MulticastConfig();
        multicastConfig.setEnabled(false);
        joinConfig.setMulticastConfig(multicastConfig);
        TcpIpConfig tcpIpConfig = new TcpIpConfig();
        tcpIpConfig.setEnabled(true);
        List<String> members = Arrays.asList(hzNetworkMembers.split(","));
        tcpIpConfig.setMembers(members);
        joinConfig.setTcpIpConfig(tcpIpConfig);
        networkConfig.setJoin(joinConfig);

        //Finish Network
        hazelcastConfig.setNetworkConfig(networkConfig);
        clusterManager = new HazelcastClusterManager(hazelcastConfig);


    VertxOptions options = new VertxOptions().setClusterManager(clusterManager);
    options.setClusterHost(interfaces.get(0));
    options.setMaxWorkerExecuteTime(VertxOptions.DEFAULT_MAX_WORKER_EXECUTE_TIME * workerVerticleMaxExecutionTime);
    options.setBlockedThreadCheckInterval(1000 * 60 * 60);

    Vertx.clusteredVertx(options, res -> {
        if (res.succeeded()) {
            vertx = res.result();
        } else {
            throw new RuntimeException("Unable to launch Vert.x");
        }
    });

********* Alternate Solution **********

we actually changed our distributed caching implementation from hazelcast to Redis (Amazon ElastiCache).

We coudnt rely on hazelcast for 3 reasons.

1) because of its inconsistency during server restarts
2) we were using embedded hazelcast and we ended up restarting our app when hazelcast data in inconsistent and we want our app to be independent of other services
3) memory allocation (hazelcast data) now is independent of application server

Upvotes: 2

Views: 1967

Answers (1)

Jayson Minard
Jayson Minard

Reputation: 85946

Vertx 3.2.0 now supports handing it a preconfigured Hazelcast instance for which to build a cluster. Therefore you have complete control over the Hazelcast configuration including how and where you want data stored. But you also need a bug fix from Vert.x 3.2.1 release to really use this.

See updated documentation at https://github.com/vert-x3/vertx-hazelcast/blob/master/src/main/asciidoc/index.adoc#using-an-existing-hazelcast-cluster

Note: When you create your own cluster, you need to have the extra Hazelcast settings required by Vertx. And those are noted in the documentation above.

Vert.x 3.2.1 release fixes an issue that blocks the use of client connections. Be aware that if you do distributed locks with Hazelcast clients, the default timeout is 60 seconds for the lock to go away if the network connection is stalled in a way that isn't obvious to the server nodes (all other JVM exits should immediately clear a lock).

You can lower this amount using:

// This is checked every 10 seconds, so any value < 10 will be treated the same
System.setProperty("hazelcast.client.max.no.heartbeat.seconds", "9");

Also be aware that with Hazelcast clients you may want to use near caching for some maps and look at other advanced configuration options for performance tuning a client which will behave differently than a full data node.

Since version 3.2.1 you can run other full Hazelcast nodes configured correctly with the map settings required by Vertx. And then create custom Hazelcast clients when starting Vertx (taken from a new unit test case):

ClientConfig clientConfig = new ClientConfig().setGroupConfig(new GroupConfig("dev", "dev-pass"));
HazelcastInstance clientNode1 = HazelcastClient.newHazelcastClient(clientConfig);

HazelcastClusterManager mgr1 = new HazelcastClusterManager(clientNode1);
VertxOptions options1 = new VertxOptions().setClusterManager(mgr1).setClustered(true).setClusterHost("127.0.0.1");

Vertx.clusteredVertx(options1, ...)

Obviously your client configuration and needs will differ. Consult the Hazelcast documentation for Client configuration: http://docs.hazelcast.org/docs/3.5/manual/html-single/index.html

Upvotes: 1

Related Questions