Apache Mina sshd: kex_exchange_identification: read: Connection reset by peer

Question

Good evening,

currently trying to setup a simple (starting point) ssh server with apache mina inside a quarkus application.

My current code looks like this (yes the logging is on purpose for now):

    public void start() throws IOException {
        if (sshConfig.disable()) {
            return;
        }
        if (sshd != null) {
            return;
        }

        sshd = org.apache.sshd.server.SshServer.setUpDefaultServer();

        sshd.setPort(sshConfig.port());
        sshd.setHost(sshConfig.listenAddress());

        // If the host key does not exist yet, it will be auto-created
        final File hostKey = new File(sshConfig.privateHostKey());
        SimpleGeneratorHostKeyProvider hostKeyProvider = new SimpleGeneratorHostKeyProvider(hostKey.toPath());
        hostKeyProvider.setAlgorithm(KeyUtils.RSA_ALGORITHM);
        if (!hostKey.exists()) {
            final List<KeyPair> hostKeys = hostKeyProvider.loadKeys(null);
            if (!hostKeys.isEmpty()) {
                Files.writeString(
                        new File(sshConfig.privateHostKey()).toPath(),
                        hostKeys.getFirst().getPrivate().toString()
                );
                Files.writeString(
                        new File(sshConfig.publicHostKey()).toPath(),
                        hostKeys.getFirst().getPublic().toString()
                );
            }
        }
        sshd.setKeyPairProvider(hostKeyProvider);
        sshd.setHostKeyCertificateProvider(new FileHostKeyCertificateProvider(hostKey.toPath()));

        sshd.setFileSystemFactory(NativeFileSystemFactory.INSTANCE);
        sshd.setShellFactory(channel -> {
            System.out.println("ShellFactory: " + channel);
            return new InteractiveProcessShellFactory().createShell(channel);
        });
        sshd.setPasswordAuthenticator(new LoggingPasswordAuthenticator());
        sshd.setKeyboardInteractiveAuthenticator(new LoggingKeyboardInteractiveAuthenticator());
        sshd.setPublickeyAuthenticator(new LoggingPublicKeyAuthenticator());
        sshd.setGSSAuthenticator(null);
        sshd.setHostBasedAuthenticator((session, username, clientHostKey, clientHostName, clientUsername, certificates) -> {
            System.out.println("Username: " + username);
            System.out.println("ClientHostKey: " + clientHostKey);
            System.out.println("ClientHostName: " + clientHostName);
            System.out.println("ClientUsername: " + clientUsername);
            return true;
        });

        sshd.setSubsystemFactories(List.of(
                        new SftpSubsystemFactory()
        ));
        sshd.setCommandFactory((channel, command) -> {
            System.out.println("Command: " + command);
            return new UnknownCommand(command);
        });

//        sshd.setSessionFactory(new NoStrictKexSessionFactory(sshd));

        sshd.start();
    }

My used dependencies in pom.xml:

        <!-- SSH -->
        <dependency>
            <groupId>org.apache.sshd</groupId>
            <artifactId>sshd-core</artifactId>
            <version>2.14.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.sshd</groupId>
            <artifactId>sshd-git</artifactId>
            <version>2.14.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.sshd</groupId>
            <artifactId>sshd-sftp</artifactId>
            <version>2.14.0</version>
        </dependency>

Debug log inside application:

2024-10-27 21:34:02,297 INFO  [org.apa.ssh.com.uti.sec.bou.BouncyCastleSecurityProviderRegistrar] (Quarkus Main Thread) getOrCreateProvider(BC) created instance of org.bouncycastle.jce.provider.BouncyCastleProvider
2024-10-27 21:34:02,299 DEBUG [org.apa.ssh.com.uti.sec.SecurityUtils] (Quarkus Main Thread) register(EdDSA) not registered - enabled=true, supported=false
2024-10-27 21:34:02,322 INFO  [org.apa.ssh.com.io.DefaultIoServiceFactoryFactory] (Quarkus Main Thread) No detected/configured IoServiceFactoryFactory; using Nio2ServiceFactoryFactory
2024-10-27 21:34:02,326 DEBUG [org.apa.ssh.ser.SshServer] (Quarkus Main Thread) start() - resolve bind host=127.0.0.1
2024-10-27 21:34:02,326 DEBUG [org.apa.ssh.com.io.nio.Nio2Acceptor] (Quarkus Main Thread) bind(/127.0.0.1:2222) binding to address
2024-10-27 21:34:02,328 DEBUG [org.apa.ssh.com.io.nio.Nio2Acceptor] (Quarkus Main Thread) setOption(SO_REUSEADDR)[true] from property=Property[socket-reuseaddr](Boolean]
2024-10-27 21:34:02,328 DEBUG [org.apa.ssh.com.io.nio.Nio2Acceptor] (Quarkus Main Thread) bind(/127.0.0.1:2222) bound to /127.0.0.1:2222
2024-10-27 21:34:02,367 INFO  [io.quarkus] (Quarkus Main Thread) git-server 1.0.0-SNAPSHOT on JVM (powered by Quarkus 3.15.1) started in 1.022s. Listening on: http://localhost:8080
2024-10-27 21:34:02,367 INFO  [io.quarkus] (Quarkus Main Thread) Profile dev activated. Live Coding activated.
2024-10-27 21:34:02,367 INFO  [io.quarkus] (Quarkus Main Thread) Installed features: [cdi, config-yaml, rest, rest-jackson, smallrye-context-propagation, vertx]
2024-10-27 21:34:04,362 DEBUG [org.apa.ssh.com.io.nio.Nio2Acceptor] (sshd-SshServer[2fa1216](port=2222)-nio2-thread-1) setOption(SO_REUSEADDR)[true] from property=Property[socket-reuseaddr](Boolean]
2024-10-27 21:34:04,364 DEBUG [org.apa.ssh.com.io.nio.Nio2Session] (sshd-SshServer[2fa1216](port=2222)-nio2-thread-1) Creating IoSession on /127.0.0.1:2222 from /127.0.0.1:55728 via /127.0.0.1:2222

And the ssh command I am running: ssh -vvvv -i keyFile -p 2222 admin@127.0.0.1

OpenSSH_9.6p1 Ubuntu-3ubuntu13.5, OpenSSL 3.0.13 30 Jan 2024
debug1: Reading configuration data /home/USER/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolve_canonicalize: hostname 127.0.0.1 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/USER/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/USER/.ssh/known_hosts2'
debug3: channel_clear_timeouts: clearing
debug3: ssh_connect_direct: entering
debug1: Connecting to 127.0.0.1 [127.0.0.1] port 2222.
debug3: set_sock_tos: set socket 3 IP_TOS 0x10
debug1: Connection established.
debug1: identity file keyFile type 0
debug1: identity file keyFile-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.5
kex_exchange_identification: read: Connection reset by peer
Connection reset by 127.0.0.1 port 2222

When debugging I can debug to the point, where mina wants to getServer() for the new session, but stepping into getServer() immediately throws an exception without any reason and even without stepping into the method a few lines above:

Tried to copy the test code from https://github.com/apache/mina-sshd/blob/master/sshd-core/src/test/java/org/apache/sshd/util/test/CoreTestSupportUtils.java#L86 and modified a lot based on this code.

• Tried serverBuilder withevery permutation of parameters
• Tried to setup default server and modify parameters afterwards
• Tried to set welcome banner (as kex_identification errors normally happen, when no version number is sent or handshake errors happen)
• Tried to setup NoStrictKexSession from https://github.com/apache/mina-sshd/blob/master/docs/howto.md
• Tried to login with username/password, public RSA key, public ed25519 key
• Tried to generate different host keys
• Tried to setup manually generated host keys

Any hint, or advice would be really useful since I am lacking ideas what to try next.

Best regards

Just came to my mind: Maybe Quarkus is intercepting all calls to any port? Wouldn't know how as quarkus has no privileges on the opened socket by mina, but maybe that's a possible direction? If so, how to tell quarkus to not to do so?