CODEWITHSUNDEEP
ubuntu-14.04ipv6rheljgroups
eminemence
eminemence

Reputation: 781

jGroups does not join any group on RHEL with link-local ipv6 address

Am seeing this exception in RHEL 7.3.1 when running my code to form a jGroups cluster. The following exception is seen in log.

[DEBUG] 2017-10-03 20:23:01.339 [pool-10-thread-1] client.jgroups  - Creating new Channel
[WARN ] 2017-10-03 20:23:01.342 [pool-10-thread-1] stack.Configurator  - JGRP000014: TP.loopback has been deprecated: enabled by default
[DEBUG] 2017-10-03 20:23:01.343 [pool-10-thread-1] stack.Configurator  - set property UDP.bind_addr to default value /fe80:0:0:0:2d57:389e:e4fe:9520%eth0
[DEBUG] 2017-10-03 20:23:01.345 [pool-10-thread-1] stack.Configurator  - set property UDP.diagnostics_addr to default value /ff0e:0:0:0:0:0:75:75
[DEBUG] 2017-10-03 20:23:01.346 [pool-10-thread-1] client.jgroups  - STATE OPEN
[DEBUG] 2017-10-03 20:23:01.347 [pool-10-thread-1] protocols.UDP  - sockets will use interface fe80:0:0:0:2d57:389e:e4fe:9520%eth0
[ERROR] 2017-10-03 20:23:01.374 [pool-10-thread-1] client.jgroups  - Catching
java.lang.Exception: failed to open a port in range 40000-40255
    at org.jgroups.protocols.UDP.createDatagramSocketWithBindPort(UDP.java:500) ~[xxx-xxx.jar:2.0.1]
    at org.jgroups.protocols.UDP.createSockets(UDP.java:361) ~[xxx-xxx.jar:2.0.1]
    at org.jgroups.protocols.UDP.start(UDP.java:270) ~[xxx-xxx.jar:2.0.1]
    at org.jgroups.stack.ProtocolStack.startStack(ProtocolStack.java:965) ~[xxx-xxx.jar:2.0.1]
    at org.jgroups.JChannel.startStack(JChannel.java:891) ~[xxx-xxx.jar:2.0.1]
    at org.jgroups.JChannel._preConnect(JChannel.java:553) ~[xxx-xxx.jar:2.0.1]
    at org.jgroups.JChannel.connect(JChannel.java:288) ~[xxx-xxx.jar:2.0.1]
    at org.jgroups.JChannel.connect(JChannel.java:279) ~[xxx-xxx.jar:2.0.1]

Now the same client code runs perfectly on a Ubuntu 14.04 machine. Another thing to note is that the following flag is not provided in both the cases.

-Djava.net.preferIPv4Stack=true

Also in both the cases link-local IPv6 addresses are being used. How to make the same code work on RHEL?

Adding the following info, for the questions asked by @bela-ban : Trying options in config xml.

I tried both LINK_LOCAL & NON_LOOPBACK, but still getting the same error.

JGroups version?

I am using 3.6.3-Final version of JGroups.

Omitting IPv4 flag

We have omitted -Djava.net.preferIPv4Stack=true, as we want to test our client in an IPv6 client environment.

Running ifconfig -a

Also running the command ifconfig -a , gives the following output : 

ifconfig -a
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.66.194.103  netmask 255.255.252.0  broadcast 10.66.195.255
        inet6 fe80::4b16:4a66:2bc3:c505  prefixlen 64  scopeid 0x20<link>
        inet6 fe80::30cb:2f41:5e04:51c2  prefixlen 64  scopeid 0x20<link>
        inet6 fe80::2d57:389e:e4fe:9520  prefixlen 64  scopeid 0x20<link>
        ether 00:15:5d:b8:65:47  txqueuelen 1000  (Ethernet)
        RX packets 8485475  bytes 1961303302 (1.8 GiB)
        RX errors 0  dropped 109087  overruns 0  frame 0
        TX packets 49088  bytes 4169469 (3.9 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 154252  bytes 11261136 (10.7 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 154252  bytes 11261136 (10.7 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Upvotes: 0

Views: 267

Answers (2)

eminemence
eminemence

Reputation: 781

So this was all failing due to bad local link addresses.

The first mistake was to use the now obsolete ifconfig command. It does not provide any information if the local link addresses assigned are valid or not.

The correct command to use is ip address. This command in my case returns the following : 

# ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:15:5d:b8:65:47 brd ff:ff:ff:ff:ff:ff
    inet 10.66.194.103/22 brd 10.66.195.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::2d57:389e:e4fe:9520/64 scope link tentative dadfailed
       valid_lft forever preferred_lft forever
    inet6 fe80::30cb:2f41:5e04:51c2/64 scope link tentative dadfailed
       valid_lft forever preferred_lft forever
    inet6 fe80::4b16:4a66:2bc3:c505/64 scope link tentative dadfailed
       valid_lft forever preferred_lft forever

As you can see the ipv6 local link addresses listed here are marked as tentative dadfailed. This means that these addresses cannot be used for anything. So the next step was to weed out these bad addresses and add our own unique local address. I did the following steps to accomplish this:

#add the new unique local address. Again this can be duplicate, so chose wisely. A reboot may be required after this.
$nmcli c mod eth0 ipv6.addresses fc00::10:8:8:71/7 ipv6.method manual
# Remove out the old local link addresses
$ip address delete fe80::4b16:4a66:2bc3:c505/64 dev eth0
$ip address delete fe80::30cb:2f41:5e04:51c2/64 dev eth0
$ip address delete fe80::2d57:389e:e4fe:9520/64 dev eth0

After this we can verify again if the above steps have worked or not

ip address show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:15:5d:b8:65:47 brd ff:ff:ff:ff:ff:ff
    inet 10.66.194.103/22 brd 10.66.195.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fc00::10:8:8:71/7 scope global
       valid_lft forever preferred_lft forever

See no more tentative dadfailed.

Hence, to conclude this was not related to JGroups at all and was only caused by bad local link addresses.

Upvotes: 0

Bela Ban
Bela Ban

Reputation: 2186

Which version of JGroups do you use? (java -cp jgroups.jar org.jgroups.Version prints the version to stdout).

Using system property -Djava.net.preferIPv4Stack=true will force the use of IPv4 addresses. In your case, on RHEL you seemed to have omitted this property, therefore using IPv6 addresses.

Make sure you do have an address fe80:0:0:0:2d57:389e:e4fe:9520%eth0 (ifconfig -a). Note that you can use bind_addr=link_local to pick any link local address.

[1] http://www.jgroups.org/manual4/index.html#Transport

Upvotes: 0

Related Questions