ErrorsStaticallyReferencedJniMethods: java.lang.UnsatisfiedLinkError

Question

I am running a netty-based component and see UnsatisfiedLinkError error on attempt to bind on a bound port. Could someone probably clarify, what could be the root cause of this particular error? I read about potential compatibility issues. In order to make sure that I don't have any version conflicts I've tried netty bom, reactor bom, spring boot bom - nothing seem to resolve the issue. I've also tried different versions of netty, reactor, spring boot.

pom.xml (with reactor bom):

<dependency>
    <groupId>io.projectreactor.netty</groupId>
    <artifactId>reactor-netty-core</artifactId>
</dependency>

<dependency>
    <groupId>io.projectreactor.netty</groupId>
    <artifactId>reactor-netty-http</artifactId>
</dependency>


<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-bom</artifactId>
            <version>2022.0.21</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

netty jars in classpath:

netty-buffer-4.1.105.Final.jar
netty-codec-4.1.105.Final.jar
netty-codec-dns-4.1.105.Final.jar
netty-codec-http2-4.1.105.Final.jar
netty-codec-http-4.1.105.Final.jar
netty-codec-socks-4.1.105.Final.jar
netty-common-4.1.105.Final.jar
netty-handler-4.1.105.Final.jar
netty-handler-proxy-4.1.105.Final.jar
netty-resolver-4.1.105.Final.jar
netty-resolver-dns-4.1.105.Final.jar
netty-resolver-dns-classes-macos-4.1.105.Final.jar
netty-resolver-dns-native-macos-4.1.105.Final-osx-x86_64.jar
netty-transport-4.1.105.Final.jar
netty-transport-classes-epoll-4.1.105.Final.jar
netty-transport-native-epoll-4.1.105.Final-linux-x86_64.jar
netty-transport-native-unix-common-4.1.105.Final.jar
reactor-netty-core-1.1.15.jar
reactor-netty-http-1.1.15.jar

Stacktrace:

java.lang.UnsatisfiedLinkError: 'int io.netty.channel.unix.ErrorsStaticallyReferencedJniMethods.errorEHOSTUNREACH()'
    at io.netty.channel.unix.ErrorsStaticallyReferencedJniMethods.errorEHOSTUNREACH(Native Method) ~[netty-transport-native-unix-common-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.unix.Errors.<clinit>(Errors.java:62) ~[netty-transport-native-unix-common-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.unix.Socket.bind(Socket.java:378) ~[netty-transport-native-unix-common-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.epoll.AbstractEpollChannel.doBind(AbstractEpollChannel.java:730) ~[netty-transport-classes-epoll-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.epoll.EpollServerSocketChannel.doBind(EpollServerSocketChannel.java:76) ~[netty-transport-classes-epoll-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:562) ~[netty-transport-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334) ~[netty-transport-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:600) ~[netty-transport-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:579) ~[netty-transport-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973) ~[netty-transport-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:260) ~[netty-transport-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:380) ~[netty-transport-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173) ~[netty-common-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166) ~[netty-common-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:413) ~[netty-transport-classes-epoll-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.105.Final.jar!/:4.1.105.Final]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.105.Final.jar!/:4.1.105.Final]
    at java.lang.Thread.run(Thread.java:1583) ~[?:?]

Netty is able to load the required library:

D 241016 032344.803 [ServerRunner] NativeLibraryLoader {} - -Dio.netty.native.workdir: /tmp (io.netty.tmpdir)
D 241016 032344.804 [ServerRunner] NativeLibraryLoader {} - -Dio.netty.native.deleteLibAfterLoading: true
D 241016 032344.804 [ServerRunner] NativeLibraryLoader {} - -Dio.netty.native.tryPatchShadedId: true
D 241016 032344.804 [ServerRunner] NativeLibraryLoader {} - -Dio.netty.native.detectNativeLibraryDuplicates: true
D 241016 032344.815 [ServerRunner] NativeLibraryLoader {} - Successfully loaded the library netty_transport_native_epoll_x86_64

Answer 1

I have identified that the issue is environment specific. I can reproduce it on one Linux host, but it works without any problems on a different Linux host. So, on the host where I can observe the issue, for some reason, netty is unable to call the following native method: io.netty.channel.unix.ErrorsStaticallyReferencedJniMethods.errorEHOSTUNREACH. Other methods are accessible, but this single one is not.

uname -a

Linux <host-name> 6.5.0-1020-aws #20~22.04.1-Ubuntu SMP Wed May  1 16:10:50 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

cat /etc/os-release

PRETTY_NAME="Ubuntu 22.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.4 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

ldd --version

ldd (Ubuntu GLIBC 2.35-0ubuntu3.8) 2.35
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.