Apache Commons FTP Passive mode, how to set remote listening port (data stream)

Question

I try to connect with an FTP server with apache-commons-net-3.7.2 (implicit TLS, double factor authentication with client cert + login/password).

I can authenticate myself, enter in passive mode, but the client doesn't succeed in connecting to the server in order to get data by the data socket.

I can connect myself, on the same computer, with WinSCP (same settings). I have activated WinSCP logs to see protocol details, and I have adjusted my source code with the same options. I can verify that my protocol is ok with a ProtocolCommandListener. I know that passive mode is required because WinSCP emits PASV command.

I can see that WinSCP connects to the data socket on port 62564 (I have replaced FTP IP address with XXX)

2021-01-06 10:25:35.575 227 Entering Passive Mode (192,168,4,122,244,100).
2021-01-06 10:25:35.575 Server sent passive reply with unroutable address 192.168.4.122, using host address instead.
2021-01-06 10:25:35.575 MLSD
2021-01-06 10:25:35.575 Connexion à 83.XXX.XXX.XXX:62564...
2021-01-06 10:25:35.604 150 Transferring directory

Also I can see that the reply sended by the server for PASV command doesn't include the port to connect to.

public class TestApi {
    
    public static void _parseExtendedPassiveModeReply(String reply)
        {
            reply = reply.substring(reply.indexOf('(') + 1,
                    reply.indexOf(')')).trim();
    
            char delim1, delim2, delim3, delim4;
            delim1 = reply.charAt(0);
            delim2 = reply.charAt(1);
            delim3 = reply.charAt(2);
            delim4 = reply.charAt(reply.length()-1);
    
            if (!(delim1 == delim2) || !(delim2 == delim3)
                    || !(delim3 == delim4)) {
                System.out.println("Could not parse extended passive host information.\nServer Reply: " + reply);
            }
    
            int port;
            try
            {
                port = Integer.parseInt(reply.substring(3, reply.length()-1));
            }
            catch (NumberFormatException e)
            {
                System.out.println("Could not parse extended passive host information.\nServer Reply: " + reply);
            }
        }   
    
    public static void main(String[] args) throws SocketException, IOException, GeneralSecurityException {

        String hostname = args[0];
        int port = Integer.parseInt(args[1]);
        String login = args[2];
        String pwd = args[3];

        FTPSClient client = new FTPSClient("TLS",true);
        
        File clientCertStore = new File("myJCEKS keystore");        
        KeyManager keyManager = KeyManagerUtils.createClientKeyManager("JCEKS",clientCertStore,"","myalias","");        
        client.setKeyManager(keyManager);
        
        client.connect(hostname, port);
        int reply = client.getReplyCode();

        if (!FTPReply.isPositiveCompletion(reply)) {
            client.disconnect();
            System.err.println("FTP server refused connection.");
            System.exit(1);
        } else {
            if (client.login(login, pwd)) {
                
                client.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.err), true));
                client.sendCommand("OPTS","UTF8 ON");
                client.sendCommand("PBSZ","0");
                client.sendCommand("PROT","P");
                
                int retour = client.pasv();
                System.out.println(retour);
                _parseExtendedPassiveModeReply(client.getReplyString());                
                
                System.out.println(client.printWorkingDirectory());
                reply = client.getReplyCode();
                System.out.println(reply);
                
                System.out.println(client.listHelp());
                
                //it freezes here, after sending MLDS command 
                //same thing using regular api for listing files (WinSCP use MLSD while regular api uses LIST)      
                client.sendCommand("MLSD");

                //and so on

                System.out.println("LOGOUT");
                client.logout();
            } else {
                System.out.println("echec login");
            }
        }

    }
}

I suppose that the API doesn't know on which port data request have to be sended and use a default which is not ok. I don't know how WinSCP succeeds in computing 62564 port number.

OPTS UTF8 ON
200 Command OPTS succeed
PBSZ 0
200 PBSZ=0
PROT P
200 PRIVATE data channel protection level set
PASV
227 Entering Passive Mode (192,168,4,122,245,74).
227
PWD
Could not parse extended passive host information.
Server Reply: 192,168,4,122,245,74
Could not parse extended passive host information.
Server Reply: 192,168,4,122,245,74
257 "/" is current directory
/
257
HELP
214-The following commands are implemented
    ABOR  ACCT  ALLO* APPE  CDUP  CWD   DELE  FEAT+ HELP
    HOST+ LANG+ LIST  MDTM+ MLST+ MKD   MODE  NLST  NOOP
    OPTS+ PASS  PASV  PORT  PWD   QUIT  REIN  REST  RETR
    RMD   RNFR  RNTO  SITE  SIZE  SMNT  STAT  STOR  STOU
    STRU* SYST  TYPE  USER  XCUP  XCRC  XCWD  XMD5  XMKD
    XPWD  XRMD  XSIGN XSHA1 XSHA256 XSHA512 XQUOTA
214 Help complete
214-The following commands are implemented
    ABOR  ACCT  ALLO* APPE  CDUP  CWD   DELE  FEAT+ HELP
    HOST+ LANG+ LIST  MDTM+ MLST+ MKD   MODE  NLST  NOOP
    OPTS+ PASS  PASV  PORT  PWD   QUIT  REIN  REST  RETR
    RMD   RNFR  RNTO  SITE  SIZE  SMNT  STAT  STOR  STOU
    STRU* SYST  TYPE  USER  XCUP  XCRC  XCWD  XMD5  XMKD
    XPWD  XRMD  XSIGN XSHA1 XSHA256 XSHA512 XQUOTA
214 Help complete

MLSD

After hours searching in API documentation, source code, FTP RFC, I don't see how to do it.

Answer 1

Your assumption is wrong. You do not set the port. The server tells you what port to connect to.

For WinSCP:

2021-01-06 10:25:35.575 227 Entering Passive Mode (192,168,4,122,244,100).
...
2021-01-06 10:25:35.575 Connexion à 83.XXX.XXX.XXX:62564...

Where 62564 = (244 << 8) + 100

See RFC 959, section 4.1.2. Transfer parameter commands, Page 28.

---

The parsing of the PASV response fails, because you are using a wrong code. The _parseExtendedPassiveModeReply is for EPSV. For PASV, use _parsePassiveModeReply. There you will also see the implementation of the above formula:

int oct1 = Integer.parseInt(m.group(2));
int oct2 = Integer.parseInt(m.group(3));
__passivePort = (oct1 << 8) | oct2;