CODEWITHSUNDEEP
hazelcast
yoyo
yoyo

Reputation: 3

Hazelcast 3.6 - java.io.IOException: No available connection to address

i am using hazelcast 3.6 cluster consist of 2 nodes . my client configuration is :

ClientConfig clientConfig = new ClientConfig();
    clientConfig.getGroupConfig().setName("dev").setPassword("dev-pass");

    String[] list = hazelcastServerList.toString().split(" ");

    clientConfig.getNetworkConfig().addAddress(list);
    clientConfig.getNetworkConfig().setConnectionAttemptLimit(5);
    clientConfig.getNetworkConfig().setSmartRouting(true);
    HazelcastInstance client = HazelcastClient.newHazelcastClient(clientConfig);

i see that sometimes i get this error :

error is :java.io.IOException: No available connection to address Address[{node1_address}]:5701

i wonder :

  1. why it happens
  2. why it is not failed over to the second node, this is the whole purpose of the cluster , isn't it ?

i don't know if it is related or not , but the address of the hazelcast servers is recognized behind a VPN network , and is resolved to a private IP.

member config is : 

Copyright (C) 2012.
Olaf Bergner.
Hamburg, Germany. [email protected]
All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an "AS IS"
BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
express or implied. See the License for the specific language
governing permissions and limitations under the License.
 -->

<hazelcast
xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.6.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
id="hazelcast-server.defaultInstance">

<properties>
    <property
        name="hazelcast.logging.type">slf4j</property>
    <property
        name="hazelcast.version.check.enabled">false</property>
     <property
        name="hazelcast.mancenter.enabled">false</property>
    <property
        name="hazelcast.memcache.enabled">true</property>
    <property
        name="hazelcast.rest.enabled">true</property>
    <property
        name="hazelcast.log.state">true</property>
    <property
        name="hazelcast.jmx">true</property>
    <property
        name="hazelcast.jmx.detailed">true</property>
    <property
        name="hazelcast.executor.client.thread.count">100</property>
</properties>

<group>
    <name>dev</name>
    <password>dev-pass</password>
</group>
<management-center
    enabled="false">http://localhost:8080/mancenter</management-center>

<network>
    <port
        auto-increment="true">5701</port>
    <join>
        <multicast
            enabled="false">
            <multicast-group>IP</multicast-group>
            <multicast-port>54327</multicast-port>
            <multicast-timeout-seconds>3</multicast-timeout-seconds>
            </multicast>
        <tcp-ip connection-timeout-seconds="60"
            enabled="true">
            <!-- <connection-timeout-seconds>60</connection-timeout-seconds> -->
            <interface>hostname1:5701</interface>
            <interface>hostname2:5701</interface>
        </tcp-ip>
    </join>
    <interfaces
        enabled="false">
        <interface>10.10.1.*</interface>
    </interfaces>
    <ssl
        enabled="false" />
    <socket-interceptor
        enabled="false" />
</network>
<partition-group
    enabled="false" />

<executor-service  name="exec">
       <pool-size>16</pool-size>
       <!--Queue capacity. 0 means Integer.MAX_VALUE.-->
    <queue-capacity>0</queue-capacity>
       <statistics-enabled>true</statistics-enabled>
    <!-- <core-pool-size>50</core-pool-size>
    <max-pool-size>200</max-pool-size>
    <keep-alive-seconds>60</keep-alive-seconds> -->

</executor-service>
<map name="default">
    <!--
        Number of backups. If 1 is set as the backup-count for example, then all entries of
        the map will be copied to another JVM for fail-safety. 0 means no backup.
    -->
    <backup-count>1</backup-count>
    <!--
            Maximum number of seconds for each entry to stay in the map. Entries that are
                    older than <time-to-live-seconds> and not updated for <time-to-live-seconds>
                    will get automatically evicted from the map.
                    Any integer between 0 and Integer.MAX_VALUE. 0 means infinite. Default is 0.
    -->
            <time-to-live-seconds>86400</time-to-live-seconds>


       <!--
            Maximum number of seconds for each entry to stay idle in the map. Entries that are
                    idle(not touched) for more than <max-idle-seconds> will get
                    automatically evicted from the map. Entry is touched if get, put or containsKey is called.
                    Any integer between 0 and Integer.MAX_VALUE. 0 means infinite. Default is 0.
    -->
    <max-idle-seconds>86400</max-idle-seconds>
    <!--
        Valid values are:
        NONE (no eviction),
        LRU (Least Recently Used),
        LFU (Least Frequently Used).
        NONE is the default.
    -->
    <eviction-policy>LFU</eviction-policy>
       <!--
        Maximum size of the map. When max size is reached,
        map is evicted based on the policy defined.
        Any integer between 0 and Integer.MAX_VALUE. 0 means
        Integer.MAX_VALUE. Default is 0.
    -->
     <max-size policy="PER_NODE">100000</max-size>
        <!--
        When max. size is reached, specified percentage of
        the map will be evicted. Any integer between 0 and 100.
        If 25 is set for example, 25% of the entries will
        get evicted.
        -->
      <eviction-percentage>15</eviction-percentage>
      <!--
        Minimum time in milliseconds which should pass before checking
        if a partition of this map is evictable or not.
        Default value is 100 millis.
       -->
    <min-eviction-check-millis>100</min-eviction-check-millis>
    <!--
        While recovering from split-brain (network partitioning),
        map entries in the small cluster will merge into the bigger cluster
        based on the policy set here. When an entry merge into the
        cluster, there might an existing entry with the same key already.
        Values of these entries might be different for that same key.
        Which value should be set for the key? Conflict is resolved by
        the policy set here. Default policy is PutIfAbsentMapMergePolicy
        There are built-in merge policies such as
        com.hazelcast.map.merge.PassThroughMergePolicy; entry will be  
        overwritten if merging entry exists for the key.
        com.hazelcast.map.merge.PutIfAbsentMapMergePolicy ; entry will be added if the merging entry doesn't exist in the cluster.
        com.hazelcast.map.merge.HigherHitsMapMergePolicy ; entry with the higher hits wins.
        com.hazelcast.map.merge.LatestUpdateMapMergePolicy ; entry with the latest update wins.
    -->
    <merge-policy>com.hazelcast.map.merge.LatestUpdateMapMergePolicy</merge-policy>
</map>


 <map name="local">
    <!--
        Number of backups. If 1 is set as the backup-count for example,
        then all entries of the map will be copied to another JVM for
        fail-safety. Valid numbers are 0 (no backup), 1, 2, 3.
    -->
    <backup-count>1</backup-count>

    <!--
        Maximum number of seconds for each entry to stay in the map. Entries 
        that are
        older than <time-to-live-seconds> and not updated for <time-to-live-
        seconds>
        will get automatically evicted from the map.
        Any integer between 0 and Integer.MAX_VALUE. 0 means infinite. 
        Default is 0.
    -->
    <time-to-live-seconds>86400</time-to-live-seconds>

    <!--
        Maximum number of seconds for each entry to stay idle in the map. Entries that are
        idle(not touched) for more than <max-idle-seconds> will get
        automatically evicted from the map.
        Entry is touched if get, put or containsKey is called.
        Any integer between 0 and Integer.MAX_VALUE.
        0 means infinite. Default is 0.

       -->
    <max-idle-seconds>86400</max-idle-seconds>

    <!--
        Valid values are:
        NONE (no extra eviction, <time-to-live-seconds> may still apply),
        LRU  (Least Recently Used),
        LFU  (Least Frequently Used).
        NONE is the default.
        Regardless of the eviction policy used, <time-to-live-seconds> will still apply.
    -->
    <eviction-policy>LRU</eviction-policy>

    <!--
        Maximum size of the map. When max size is reached,
        map is evicted based on the policy defined.
        Any integer between 0 and Integer.MAX_VALUE. 0 means
        Integer.MAX_VALUE. Default is 0.
    -->
  <!--  <max-size policy="cluster_wide_map_size">0</max-size> -->
  <max-size policy="PER_NODE">100000</max-size>
         <!--
        When max. size is reached, specified percentage of
        the map will be evicted. Any integer between 0 and 100.
        If 25 is set for example, 25% of the entries will
        get evicted.
    -->
    <eviction-percentage>15</eviction-percentage>
        <!--
        Specifies when eviction will be started. Default value is 3.
       So every 3 (+up to 5 for performance reasons) seconds
       eviction will be kicked of. Eviction is costly operation, setting
       this number too low, can decrease the performance. -->
     <!--
        Minimum time in milliseconds which should pass before checking
        if a partition of this map is evictable or not.
        Default value is 100 millis.
    -->
      <min-eviction-check-millis>100</min-eviction-check-millis>
      <!--
        While recovering from split-brain (network partitioning),
        map entries in the small cluster will merge into the bigger cluster
        based on the policy set here. When an entry merge into the
        cluster, there might an existing entry with the same key already.
        Values of these entries might be different for that same key.
        Which value should be set for the key? Conflict is resolved by
        the policy set here. Default policy is PutIfAbsentMapMergePolicy
        There are built-in merge policies such as
        com.hazelcast.map.merge.PassThroughMergePolicy; entry will be 
         overwritten if merging entry exists for the key.
        com.hazelcast.map.merge.PutIfAbsentMapMergePolicy ; entry will be 
        added if the merging entry doesn't exist in the cluster.
        com.hazelcast.map.merge.HigherHitsMapMergePolicy ; entry with the 
        higher hits wins.
        com.hazelcast.map.merge.LatestUpdateMapMergePolicy ; entry with the 
        latest update wins.
    -->
   <merge-policy>com.hazelcast.map.merge.LatestUpdateMapMergePolicy</merge-
      policy>
     </map>

    </hazelcast>

Upvotes: 0

Views: 1651

Answers (1)

Vik Gamov
Vik Gamov

Reputation: 5456

In your member config, you need to change tcp joiner config like

<tcp-ip connection-timeout-seconds="60" enabled="true"> <!--connection-timeout-seconds>60</connection-timeout-seconds --> <member>hostname1:5701</member> <member>hostname2:5701</member> </tcp-ip> In this case, client config should look like 

    ClientConfig clientConfig = new ClientConfig();
    // those are default values, it's not necessary to explicitly set it
    clientConfig.getGroupConfig().setName("dev").setPassword("dev-pass");

    String hazelcastServerList = "hostname1:5701 hostname2:5701";
    String[] list = hazelcastServerList.split(" ");

    clientConfig.getNetworkConfig().addAddress(list);
    clientConfig.getNetworkConfig().setConnectionAttemptLimit(5);

    // enabled by default
    clientConfig.getNetworkConfig().setSmartRouting(true);
    HazelcastInstance client = HazelcastClient.newHazelcastClient(clientConfig);

p.s. for the best performance, a client and the members should be on the same local network. To understand different ways to configure which network interfaces Hazelcast will use / listen, kindly, consult with documentation

Best,

Vik

p.p.s if you have any questions, write them in comments below.

Upvotes: 0

Related Questions