Using the non default openssl engine with netty-tcnative

Question

Question 1) How do I get netty-tcnative working with an openssl engine that is not the default one? In my case I am trying the cloudhsm engine. Has anyone any experience of loading this engine or indeed any other engine in a similar fashion. I documented the issues I am running into below.

Question 2) Should the openssl version reported by netty-tcnative on initialisation in the logs match the native openssl install version? I would expect it to however its not in my case. Perhaps this is relevant I am not sure.

More Detail I have a docker container that can access an AWS cloudHSM. I can use openssl and cloudhsm in the container using openssl on the command line.

[root@1f094ffd14e4 ob-shell]# openssl engine
(rdrand) Intel RDRAND engine
(dynamic) Dynamic engine loading support

[root@1f094ffd14e4 ob-shell]# openssl engine cloudhsm
(cloudhsm) CloudHSM hardware engine support

Everything works as I would expect it to. The native openssl version is 1.0.2k-fips

[root@1f094ffd14e4 ob-shell]# openssl version
OpenSSL 1.0.2k-fips  26 Jan 2017

I require a Java Application using netty-tcnative to access the HSM using openssl. I can't use the JCE approach due to it not working on a JDK > JDK8. AWS do not support JDK11/JDK17.

I am using the dynamically linked version of netty-tcnative.

    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-tcnative</artifactId>
        <classifier>linux-x86_64-fedora</classifier>
        <version>2.0.38.Final</version>
    </dependency>

This works well and I can see the details of the netty openssl configuration when using the default engine.

2021-09-02 13:07:45.905 DEBUG 7 --- [ main] io.netty.handler.ssl.OpenSsl : Initialize netty-tcnative using engine: 'default' 2021-09-02 13:07:45.906 DEBUG 7 --- [
main] io.netty.handler.ssl.OpenSsl : netty-tcnative using native library: OpenSSL 1.0.1e-fips 11 Feb 2013

However to use the HSM I need to use the cloudhsm engine. This is where the trouble starts. When I try to enable the cloudhsm engine with the following system Property -Dio.netty.handler.ssl.openssl.engine=cloudhsm I get problems. The cloudhsm engine does not load at all.

2021-09-02 17:19:49.433 DEBUG 7 --- [ main] i.n.util.internal.NativeLibraryLoader : Successfully loaded the library /tmp/libnetty_tcnative_linux_x86_64569349996051888224.so 2021-09-02 17:19:49.434 DEBUG 7 --- [ main] io.netty.handler.ssl.OpenSsl : Initialize netty-tcnative using engine: 'cloudhsm' Error in reading the non-fips digest methods from the library.Error caching the software digest. Dynamic Engine: cloudhsm load failed 2021-09-02 17:19:49.437 DEBUG 7 --- [
main] io.netty.handler.ssl.OpenSsl : Failed to initialize netty-tcnative; OpenSslEngine will be unavailable. See https://netty.io/wiki/forked-tomcat-native.html for more information.

java.lang.Exception: This function has not been implemented on this platform at io.netty.internal.tcnative.SSL.initialize(Native Method) ~[netty-tcnative-2.0.38.Final-linux-x86_64-fedora.jar:na] at io.netty.internal.tcnative.Library.initialize(Library.java:158) ~[netty-tcnative-2.0.38.Final-linux-x86_64-fedora.jar:na] at io.netty.handler.ssl.OpenSsl.initializeTcNative(OpenSsl.java:597) ~[netty-handler-4.1.63.Final.jar:4.1.63.Final] at io.netty.handler.ssl.OpenSsl.(OpenSsl.java:153) ~[netty-handler-4.1.63.Final.jar:4.1.63.Final] at io.netty.handler.ssl.ReferenceCountedOpenSslContext.(ReferenceCountedOpenSslContext.java:207) ~[netty-handler-4.1.63.Final.jar:4.1.63.Final] at io.netty.handler.ssl.OpenSslContext.(OpenSslContext.java:36) ~[netty-handler-4.1.63.Final.jar:4.1.63.Final] at io.netty.handler.ssl.OpenSslClientContext.(OpenSslClientContext.java:191) ~[netty-handler-4.1.63.Final.jar:4.1.63.Final] at io.netty.handler.ssl.SslContext.newClientContextInternal(SslContext.java:830) ~[netty-handler-4.1.63.Final.jar:4.1.63.Final] at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:614) ~[netty-handler-4.1.63.Final.jar:4.1.63.Final] at com.me.commands.NettyCommands.experiment(NettyCommands.java:44) ~[classes/:1.0-SNAPSHOT] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na] at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na] at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:282) ~[spring-core-5.3.6.jar:5.3.6] at org.springframework.shell.Shell.evaluate(Shell.java:169) ~[spring-shell-core-2.0.0.RELEASE.jar:2.0.0.RELEASE] at org.springframework.shell.Shell.run(Shell.java:134) ~[spring-shell-core-2.0.0.RELEASE.jar:2.0.0.RELEASE] at org.springframework.shell.jline.InteractiveShellApplicationRunner.run(InteractiveShellApplicationRunner.java:84) ~[spring-shell-core-2.0.0.RELEASE.jar:2.0.0.RELEASE] at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:810) ~[spring-boot-2.4.5.jar:2.4.5] at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:800) ~[spring-boot-2.4.5.jar:2.4.5] at org.springframework.boot.SpringApplication.run(SpringApplication.java:346) ~[spring-boot-2.4.5.jar:2.4.5] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1340) ~[spring-boot-2.4.5.jar:2.4.5] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1329) ~[spring-boot-2.4.5.jar:2.4.5] at com.me.Application.main(Application.java:41) ~[classes/:1.0-SNAPSHOT] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na] at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na] at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[ob-shell/:na] at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[ob-shell/:na] at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[ob-shell/:na] at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88) ~[ob-shell/:na]

Any ideas?