Curator Leader election give connection refused error

Question

I implemented curator leader election example which is given in this site

Instead of having number of curator clients I added only one curator client as follows

public void selectLeader() {
    CuratorFramework    client = null;
    try {
        client = CuratorFrameworkFactory.newClient("localhost:2181", new ExponentialBackoffRetry(1000, 3));
        LeaderSelectorService    service = new LeaderSelectorService(client, "/leaderSelections", "LeaderElector");

        client.start();
        Thread.sleep(10000);
        service.start();
    } catch (Exception e) {
        System.out.println("error"+e);
    }finally
    {
        System.out.println("Shutting down...");
       // CloseableUtils.closeQuietly(client);
    }

}

public class LeaderSelectorService extends LeaderSelectorListenerAdapter implements Closeable {
private final String name;
private final LeaderSelector leaderSelector;

public LeaderSelectorService(CuratorFramework client, String path, String name) {
    this.name = name;

    // create a leader selector using the given path for management
    // all participants in a given leader selection must use the same path
    // ExampleClient here is also a LeaderSelectorListener but this isn't required
    leaderSelector = new LeaderSelector(client, path, this);

    // for most cases you will want your instance to requeue when it relinquishes leadership
    leaderSelector.autoRequeue();   
}

public void start() throws IOException
    {
        // the selection for this instance doesn't start until the leader selector is started
        // leader selection is done in the background so this call to leaderSelector.start() returns immediately
        leaderSelector.start();
    }
@Override
public void takeLeadership(CuratorFramework arg0) throws Exception {
    // we are now the leader. This method should not return until we want to relinquish leadership

    final int         waitSeconds = (int)(5 * Math.random()) + 1;

    System.out.println(name + " is now the leader. Waiting " + waitSeconds + " seconds...");
    //System.out.println(name + " has been leader " + leaderCount.getAndIncrement() + " time(s) before.");
    try
    {
        Thread.sleep(TimeUnit.SECONDS.toMillis(waitSeconds));
    }
    catch ( InterruptedException e )
    {
        System.err.println(name + " was interrupted.");
        Thread.currentThread().interrupt();
    }
    finally
    {
        System.out.println(name + " relinquishing leadership.\n");
    }

}

@Override
public void close() throws IOException {
    leaderSelector.close();

}

}

I have only one zookeeper instance and I am using Zookeeper 3.4.6, curator-framework 4.0.0 and curator-recipes 4.0.0. when I start the client, it connects to zookeeper and in the log I can see "State change : connected" message.

Then I wait 10s and start leader election which gives me below error repeatedly.

2017-09-06 09:34:22.727  INFO 1228 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Unable to read additional data from server sessionid 0x15e555a719d0000, likely server has closed socket, closing socket connection and attempting reconnect
2017-09-06 09:34:22.830  INFO 1228 --- [c-1-EventThread] o.a.c.f.state.ConnectionStateManager     : State change: SUSPENDED
2017-09-06 09:34:23.302  INFO 1228 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2017-09-06 09:34:23.303  INFO 1228 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Socket connection established, initiating session, client: /127.0.0.1:49594, server: localhost/127.0.0.1:2181
2017-09-06 09:34:23.305  INFO 1228 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x15e555a719d0000, negotiated timeout = 120000
2017-09-06 09:34:23.305  INFO 1228 --- [c-1-EventThread] o.a.c.f.state.ConnectionStateManager     : State change: RECONNECTED
2017-09-06 09:34:23.310  WARN 1228 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Session 0x15e555a719d0000 for server localhost/127.0.0.1:2181, unexpected error, closing socket connection and attempting reconnect

java.io.IOException: Connection reset by peer
    at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.8.0_131]
    at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.8.0_131]
    at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.8.0_131]
    at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[na:1.8.0_131]
    at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[na:1.8.0_131]
    at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:75) ~[zookeeper-3.5.3-beta.jar:3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b42864475a60]
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:363) ~[zookeeper-3.5.3-beta.jar:3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b42864475a60]
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214) ~[zookeeper-3.5.3-beta.jar:3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b42864475a60]

after some time it started to give me below error message.

org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) ~[zookeeper-3.5.3-beta.jar:3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b42864475a60]
    at org.apache.curator.framework.imps.CuratorFrameworkImpl.checkBackgroundRetry(CuratorFrameworkImpl.java:831) [curator-framework-4.0.0.jar:4.0.0]
    at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:623) [curator-framework-4.0.0.jar:4.0.0]
    at org.apache.curator.framework.imps.WatcherRemovalFacade.processBackgroundOperation(WatcherRemovalFacade.java:152) [curator-framework-4.0.0.jar:4.0.0]
    at org.apache.curator.framework.imps.GetConfigBuilderImpl$2.processResult(GetConfigBuilderImpl.java:222) [curator-framework-4.0.0.jar:4.0.0]
    at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590) [zookeeper-3.5.3-beta.jar:3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b42864475a60]
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:499) [zookeeper-3.5.3-beta.jar:3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b42864475a60]

2017-09-06 09:34:31.897  INFO 1228 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2017-09-06 09:34:31.898  INFO 1228 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Socket connection established, initiating session, client: /127.0.0.1:49611, server: localhost/127.0.0.1:2181
2017-09-06 09:34:31.899  INFO 1228 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x15e555a719d0000, negotiated timeout = 120000
2017-09-06 09:34:31.899  INFO 1228 --- [c-1-EventThread] o.a.c.f.state.ConnectionStateManager     : State change: RECONNECTED
2017-09-06 09:34:31.907  WARN 1228 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Session 0x15e555a719d0000 for server localhost/127.0.0.1:2181, unexpected error, closing socket connection and attempting reconnect

java.io.IOException: Xid out of order. Got Xid 41 with err -6 expected Xid 40 for a packet with details: clientPath:/leaderSelections serverPath:/leaderSelections finished:false header:: 40,12  replyHeader:: 0,0,-4  request:: '/leaderSelections,F  response:: v{} 
    at org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:892) ~[zookeeper-3.5.3-beta.jar:3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b42864475a60]
    at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:101) ~[zookeeper-3.5.3-beta.jar:3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b42864475a60]
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:363) ~[zookeeper-3.5.3-beta.jar:3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b42864475a60]
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214) ~[zookeeper-3.5.3-beta.jar:3.5.3-beta-8ce24f9e675cbefffb8f21a47e06b42864475a60]

I tried several solution in the internet but non got succeeded. Does anybody know the root cause of this issue.

Answer 1

I have fixed this issue. There was version number mismatch between zookeeper version and curator version. I used curator version 4.0.0 with zookeeper 3.4.6. According to apache curator site

Curator 4.0.0 - compatible with ZooKeeper 3.5.x. I changed my curator version to 2.8.0