JGroups RPC: NoSuchMethodException

Question

Have a view consisting of 3 server nodes (American Samoa, Alaska, Alabama) and 1 client node (VoterClient). Whenever I try to invoke the vote method in the first server within the view which in the below case is American Samoa, I get a NoSuchMethodException. Here is the list of nodes in the view as shown below: (ping_dest is American Samoa, pingable_mbrs=[American Samoa, Alaska, Alabama, VoterClient]). So when I call

// Call the vote method on the state.
voteResult = dispatcher.callRemoteMethod(address1, "vote",
    new Object[] { obj.ID, obj.candidate },
    new Class[] { String.class, String.class },
    new RequestOptions(ResponseMode.GET_ALL,    50000));

where address1 is the address of "American Samoa", and obj.ID and obj.candidate are String parameters of the remote vote method, I get the NoSuchMethodException.

Here is my log and the exception:

1644 DEBUG [main]         org.jgroups.protocols.FD_SOCK     - VIEW_CHANGE received: [American Samoa, Alaska, Alabama, VoterClient, VoterClient]
1664 DEBUG [FD_SOCK pinger,StateServerGroup,VoterClient]         org.jgroups.protocols.FD_SOCK     - ping_dest is American Samoa, pingable_mbrs=[American Samoa, Alaska, Alabama, VoterClient, VoterClient]
1664 DEBUG [main]   org.jgroups.protocols.pbcast.STABLE     - [ergonomics] setting max_bytes to 20MB (5 members)
This should be occuring right?????
State server address: American Samoa.
java.lang.NoSuchMethodException: vote
    at org.jgroups.blocks.MethodCall.invoke(MethodCall.java:312)
    at org.jgroups.blocks.RpcDispatcher.handle(RpcDispatcher.java:326)
    at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:456)
    at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:363)
    at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:238)
    at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:543)
    at org.jgroups.JChannel.up(JChannel.java:716)
    at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1026)
    at org.jgroups.protocols.pbcast.STATE_TRANSFER.up(STATE_TRANSFER.java:178)
    at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)
    at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
    at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
    at org.jgroups.protocols.pbcast.GMS.up(GMS.java:889)
...
    at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1728)
    at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1710)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
failed on dispatcher for VoteAction.

Here is the vote method:

public class EVoterServer extends ReceiverAdapter implements Runnable{
    private static final String serverProperties = null; // default properties
    private static Logger logger = Logger.getLogger(EVoterServer.class);
    private JChannel channel = null;
    private String stateName = null;    
    private static final String serverProperties = null; // default properties
    private static final String channelName = "StateServerGroup";
    private boolean isRunning = true;

    @Override
    public void run() {
        try {
               channel = new JChannel(serverProperties);
            channel.setName(stateName);
            @SuppressWarnings("unused")
            RpcDispatcher dispatcher = new RpcDispatcher(channel, this, this, this);
            channel.connect(channelName);
            channel.getState(null, 0);

            logger.info(stateName + " server started at " + new Date() + ".");
            logger.info(stateName + " joined channel '" + channelName + "' (" + channel.getView().size() + " members).");
            logger.info(stateName + " server ready to serve requests.");
            logger.info(stateName + " server's channel address is " + channel.getAddress() + ".");

            isRunning = true;

            while(isRunning) {
                Util.sleep(10000);
            }
        }
        catch(Exception e) {
            logger.error("EVoterServer.run(); State: " + stateName + "; " + e.getMessage(), e);
        }
        finally
        {
            Util.close(channel);
        }
    }

    public boolean vote(String voterId, String candidateName)
    {
         System.out.println("Vote for " + candidateName + " by " + voterId + ".");
         return true;
    }
}

Here is the driver class' (main method snippet); how I initialize the servers for each state.

String [] STATE_NAMES = {"Alabama", "Alaska", "American Samoa"};
       for (int i = 0; i < 3; i++) {
           try {
              EVoterServer server = new EVoterServer(STATE_NAMES[i]);
              new Thread(server).start();
            } catch (Throwable t) {
               logger.error(t.getMessage(), t);
            }
       }

I think it has to do with the coordinator elected by the GMS. The client can't vote when the coordinator is the same as the voting server state. Below shows how the coordinator American Samoa is elected:

1401 DEBUG [main]             org.jgroups.protocols.UDP     - created unicast receiver thread 
1401 DEBUG [main]             org.jgroups.protocols.UDP     - created multicast receiver thread 
1411 DEBUG [main]      org.jgroups.protocols.pbcast.GMS     - election results: {American Samoa=2}
1411 DEBUG [main]      org.jgroups.protocols.pbcast.GMS     - sending JOIN(VoterClient) to American Samoa
1523 DEBUG [main]   org.jgroups.protocols.pbcast.NAKACK     - 
[setDigest()]

Answer 1

Since you're using the default configuration, it means that you rely on IP multicast. Multicast to self is sometimes disabled, sometimes has weird behavior - depending on the OS and the switch.

Try instantiating the JChannel with a TCP stack without multicasting (just to test the theory - TCP is an almost sure-fire). If it works, work your way down back to UDP (first without multicasting, then with). It may be that you'll have to understand if you need to do a remote call or not, depending on the address (if address == local just invoke the method normally, not remotely).

Answer 2

java.lang.NoSuchMethodException: vote

Jgroups throws that exception from MethodCall.invoke() if it was unable to find the method in question. I see a number of reasons why this could be happening:

• You mentioned that this is working on your other servers. Could it be that the one is running an older version of the software? Maybe the vote() method was updated recently?
• Any chance the EVoterServer handler object has not been registered appropriately?
• I'm curious about the error message failed on dispatcher for VoteAction. What is VoteAction? Any chance it was registered instead of EVoterServer? Maybe that's a different channel by accident that is attached to the wrong group?

Hope something here helps.