docker build fails inside gitlab-runner but works locally : spring boot native compilation with GraalVm

Question

I made a Dockerfile to build my spring boot project with GraalVm natively; everything went correctly.

Here is the Dockerfile

FROM ghcr.io/graalvm/graalvm-ce:22.3.1 AS buildnative

WORKDIR /app

COPY mvnw pom.xml ./
COPY .mvn/ .mvn
COPY src ./src
RUN ./mvnw clean package -Pnative

FROM ubuntu:23.04
EXPOSE 8080
COPY --from=buildnative /app/target/spring-boot-project /build/app
CMD ["/build/app"]

This runs perfectly locally, but in the GitLab runner, I always have the same error.

JAVA_HOME is not defined correctly.
We cannot execute /opt/graalvm-ce-java17-22.3.1/bin/java 
The command '/bin/sh -c ./mvnw clean package -Pnative' returned a non-zero code: 1

So I decided to add some logs within the maven wrapper, and here is what I have :

Step 7/11 : RUN ./mvnw clean package -Pnative ---> Running in 81e0558130f3 ------------> /opt/graalvm-ce-java17-22.3.1/bin/java ------------> JAVA_HOME is /opt/graalvm-ce-java17-22.3.1 Error: JAVA_HOME is not defined correctly. We cannot execute /opt/graalvm-ce-java17-22.3.1/bin/java The command '/bin/sh -c ./mvnw clean package -Pnative' returned a non-zero code: 1 Cleaning up project directory and file based variables

Step 7/11 : RUN ./mvnw clean package -Pnative
 ---> Running in 81e0558130f3
------------> /opt/graalvm-ce-java17-22.3.1/bin/java
------------> JAVA_HOME is /opt/graalvm-ce-java17-22.3.1
Error: JAVA_HOME is not defined correctly.
  We cannot execute /opt/graalvm-ce-java17-22.3.1/bin/java
The command '/bin/sh -c ./mvnw clean package -Pnative' returned a non-zero code: 1
Cleaning up project directory and file based variables

In the log I have added, we can see JAVA_HOME is defined and is adequately defined. It is the same as locally, where everything works perfectly.

I tried to add this line: RUN chmod +x mvnw before running it, but it did not change anything.

I need more ideas. Is there anyone have an idea of what is happening?

Edit:

I decided to dive deeper into the issue. I have added logs to know why it does not work. I modified the mvnw script to know what was happening.

I have added this to mvnw

if [ -e "$JAVACMD" ] ; then
  echo "------------> THE FILE EXIST" >&2
else
  echo "------------> THE FILE DOES NOT EXIST" >&2
fi

if [ -x "$JAVACMD" ] ; then
  echo "------------> THE FILE IS EXECUTABLE" >&2
else
  echo "------------> THE FILE IS NOT EXECUTABLE" >&2
fi

Results:

Here is in local:

------------> JAVACMD /opt/graalvm-ce-java17-22.3.1/bin/java 
------------> THE FILE EXIST 
------------> THE FILE IS EXECUTABLE 

Here is in the gitlab-runner:

------------> JAVACMD /opt/graalvm-ce-java17-22.3.1/bin/java
------------> THE FILE EXIST 
------------> THE FILE IS NOT EXECUTABLE 

Makes no sense to me

Answer 1

Never tried gitlab runner myself, but have you tried to force the JAVA_HOME env path before maven command?

ENV JAVA_HOME=/opt/graalvm-ce-java17-22.3.1
RUN ./mvnw clean package -Pnative

hope it helps

Answer 2

Is your GitLab runner configured to use a non-root user when executing the Dockerfile?

As @jilliss pointed out, it seems that it's the Java binary that needs execute permission, but maybe only root has the permission (which is why it works locally as by default you will be running it as root).

If the Ops team have tried to run the Dockerfile as another user, then it could explain why /opt/graalvm-ce-java17-22.3.1/bin/java is no longer executable.

Try adding a whoami log and see which user is running when it runs in GL.

Answer 3

Correct compile command is

mvn -Pnative native:compile

You can see more details and full doc here. After that you will see graalvm build result in the logs. So you need to change your docker file like below

FROM ghcr.io/graalvm/graalvm-ce:22.3.1 AS buildnative

WORKDIR /app

COPY mvnw pom.xml ./
COPY .mvn/ .mvn
COPY src ./src
RUN ./mvnw native:compile -Pnative

FROM ubuntu:23.04
EXPOSE 8080
COPY --from=buildnative /app/target/spring-boot-project /build/app
CMD ["/build/app"]

example build log from my local build

[INFO] --- native-maven-plugin:0.9.19:compile (default-cli) @ demo ---
Downloading: Component catalog from www.graalvm.org
Processing Component: Native Image
Downloading: Component native-image: Native Image from github.com
Installing new component: Native Image (org.graalvm.native-image, version 22.3.1)
[INFO] Found GraalVM installation from JAVA_HOME variable.
[INFO] [graalvm reachability metadata repository for ch.qos.logback:logback-classic:1.4.5]: Configuration directory not found. Trying latest version.
[INFO] [graalvm reachability metadata repository for ch.qos.logback:logback-classic:1.4.5]: Configuration directory is ch.qos.logback/logback-classic/1.4.1
[INFO] Executing: /opt/graalvm-ce-java17-22.3.1/bin/native-image -cp /app/target/classes:/root/.m2/repository/org/springframework/spring-aop/6.0.4/spring-aop-6.0.4.jar:/root/.m2/repository/org/springframework/boot/spring-boot-starter-logging/3.0.2/spring-boot-starter-logging-3.0.2.jar:/root/.m2/repository/org/springframework/spring-context/6.0.4/spring-context-6.0.4.jar:/root/.m2/repository/org/springframework/spring-core/6.0.4/spring-core-6.0.4.jar:/root/.m2/repository/org/apache/logging/log4j/log4j-api/2.19.0/log4j-api-2.19.0.jar:/root/.m2/repository/org/springframework/spring-expression/6.0.4/spring-expression-6.0.4.jar:/root/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.19.0/log4j-to-slf4j-2.19.0.jar:/root/.m2/repository/ch/qos/logback/logback-core/1.4.5/logback-core-1.4.5.jar:/root/.m2/repository/jakarta/annotation/jakarta.annotation-api/2.1.1/jakarta.annotation-api-2.1.1.jar:/root/.m2/repository/org/springframework/spring-beans/6.0.4/spring-beans-6.0.4.jar:/root/.m2/repository/ch/qos/logback/logback-classic/1.4.5/logback-classic-1.4.5.jar:/root/.m2/repository/org/springframework/boot/spring-boot-starter/3.0.2/spring-boot-starter-3.0.2.jar:/root/.m2/repository/org/springframework/spring-jcl/6.0.4/spring-jcl-6.0.4.jar:/root/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/3.0.2/spring-boot-autoconfigure-3.0.2.jar:/root/.m2/repository/org/slf4j/jul-to-slf4j/2.0.6/jul-to-slf4j-2.0.6.jar:/root/.m2/repository/org/yaml/snakeyaml/1.33/snakeyaml-1.33.jar:/root/.m2/repository/org/springframework/boot/spring-boot/3.0.2/spring-boot-3.0.2.jar:/root/.m2/repository/org/slf4j/slf4j-api/2.0.6/slf4j-api-2.0.6.jar --no-fallback -H:Path=/app/target -H:Name=demo -H:ConfigurationFileDirectories=/app/target/graalvm-reachability-metadata/160481799c4b6c37cde925c9aebf513c32245dcf/ch.qos.logback/logback-classic/1.4.1
========================================================================================================================
GraalVM Native Image: Generating 'demo' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                    (6.6s @ 0.23GB)
 Version info: 'GraalVM 22.3.1 Java 17 CE'
 Java version info: '17.0.6+10-jvmci-22.3-b13'
 C compiler: gcc (redhat, x86_64, 11.3.1)
 Garbage collector: Serial GC
 1 user-specific feature(s)
 - org.springframework.aot.nativex.feature.PreComputeFieldFeature
Field org.apache.commons.logging.LogAdapter#log4jSpiPresent set to true at build time
Field org.apache.commons.logging.LogAdapter#log4jSlf4jProviderPresent set to true at build time
Field org.apache.commons.logging.LogAdapter#slf4jSpiPresent set to true at build time
Field org.apache.commons.logging.LogAdapter#slf4jApiPresent set to true at build time
Field org.springframework.core.NativeDetector#imageCode set to true at build time
Field org.springframework.format.support.DefaultFormattingConversionService#jsr354Present set to false at build time
Field org.springframework.core.KotlinDetector#kotlinPresent set to false at build time
Field org.springframework.core.KotlinDetector#kotlinReflectPresent set to false at build time
Field org.springframework.cglib.core.AbstractClassGenerator#imageCode set to true at build time
Field org.springframework.boot.logging.log4j2.Log4J2LoggingSystem$Factory#PRESENT set to false at build time
Field org.springframework.boot.logging.java.JavaLoggingSystem$Factory#PRESENT set to true at build time
Field org.springframework.boot.logging.logback.LogbackLoggingSystem$Factory#PRESENT set to true at build time
Field org.springframework.boot.logging.logback.LogbackLoggingSystemProperties#JBOSS_LOGGING_PRESENT set to false at build time
Field org.springframework.context.event.ApplicationListenerMethodAdapter#reactiveStreamsPresent set to false at build time
[2/7] Performing analysis...  [*******]                                                                 (60.8s @ 2.15GB)
   8,903 (88.31%) of 10,082 classes reachable
  13,147 (64.27%) of 20,456 fields reachable
  40,485 (56.88%) of 71,181 methods reachable
     365 classes,   115 fields, and 1,191 methods registered for reflection
      64 classes,    70 fields, and    55 methods registered for JNI access
       4 native libraries: dl, pthread, rt, z
[3/7] Building universe...                                                                               (8.9s @ 2.03GB)
[4/7] Parsing methods...      [***]                                                                      (9.6s @ 0.82GB)
[5/7] Inlining methods...     [***]                                                                      (3.7s @ 2.16GB)
[6/7] Compiling methods...    [*******]                                                                 (48.2s @ 1.97GB)
[7/7] Creating image...                                                                                  (5.6s @ 1.60GB)
  17.53MB (49.48%) for code area:    25,460 compilation units
  17.60MB (49.66%) for image heap:  215,829 objects and 25 resources
 312.40KB ( 0.86%) for other data
  35.44MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 packages in code area:                               Top 10 object types in image heap:
 936.87KB java.util                                            3.72MB byte[] for code metadata
 594.27KB java.lang.invoke                                     2.07MB java.lang.String
 469.21KB c.s.org.apache.xerces.internal.impl.xs.traversers    2.06MB java.lang.Class
 455.78KB java.lang                                            1.68MB byte[] for general heap data
 423.05KB com.sun.org.apache.xerces.internal.impl              1.60MB byte[] for java.lang.String
 407.79KB com.sun.crypto.provider                            765.10KB com.oracle.svm.core.hub.DynamicHubCompanion
 375.00KB org.springframework.beans.factory.support          576.38KB java.util.HashMap$Node
 371.71KB java.io                                            513.09KB int[][]
 354.25KB java.util.concurrent                               394.88KB java.lang.String[]
 328.57KB java.text                                          394.65KB byte[] for reflection metadata
  12.74MB for 405 more packages                                3.23MB for 1771 more object types
------------------------------------------------------------------------------------------------------------------------
                        12.0s (8.0% of total time) in 35 GCs | Peak RSS: 3.04GB | CPU load: 5.08
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
 /app/target/demo (executable)
 /app/target/demo.build_artifacts.txt (txt)
========================================================================================================================
Finished generating 'demo' in 2m 29s.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  03:11 min
[INFO] Finished at: 2023-01-27T18:05:23Z
[INFO] ------------------------------------------------------------------------