CODEWITHSUNDEEP
mavengitlab
Mulgard
Mulgard

Reputation: 10589

Gitlab project cant resolve a maven dependency from another project in the same gitlab group

In my dependency project I have a ci_settings.xml:

<settings xmlns="http://maven.apache.org/SETTINGS/1.1.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">
    <servers>
        <server>
            <id>gitlab-maven</id>
            <configuration>
                <httpHeaders>
                    <property>
                        <name>Job-Token</name>
                        <value>${env.CI_JOB_TOKEN}</value>
                    </property>
                </httpHeaders>
            </configuration>
        </server>
    </servers>
</settings>

And I have configured the registry in my pom.xml:

    <repositories>
        <repository>
            <id>gitlab-maven</id>
            <!--suppress UnresolvedMavenProperty -->
            <url>${env.CI_API_V4_URL}/projects/${env.CI_PROJECT_ID}/packages/maven</url>
        </repository>
    </repositories>

    <distributionManagement>
        <repository>
            <id>gitlab-maven</id>
            <!--suppress UnresolvedMavenProperty -->
            <url>${CI_API_V4_URL}/projects/${env.CI_PROJECT_ID}/packages/maven</url>
        </repository>
        <snapshotRepository>
            <id>gitlab-maven</id>
            <!--suppress UnresolvedMavenProperty -->
            <url>${CI_API_V4_URL}/projects/${env.CI_PROJECT_ID}/packages/maven</url>
        </snapshotRepository>
    </distributionManagement>

When I run the CI pipline with the following .gitlab-ci.yml:

deploy:
  image: maven:3.6-jdk-11
  script:
    - mvn deploy -s ci_settings.xml
  only:
    - main

The dependency is getting deployed to the package repository successfully.

Uploading to gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/0.0.1-SNAPSHOT/model-0.0.1-20211216.121731-6.jar
Uploaded to gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/0.0.1-SNAPSHOT/model-0.0.1-20211216.121731-6.jar (35 kB at 61 kB/s)
Uploading to gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/0.0.1-SNAPSHOT/model-0.0.1-20211216.121731-6.pom
Uploaded to gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/0.0.1-SNAPSHOT/model-0.0.1-20211216.121731-6.pom (3.8 kB at 6.3 kB/s)
Downloading from gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/maven-metadata.xml
Downloaded from gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/maven-metadata.xml (283 B at 1.6 kB/s)
Uploading to gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/0.0.1-SNAPSHOT/maven-metadata.xml
Uploaded to gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/0.0.1-SNAPSHOT/maven-metadata.xml (773 B at 1.3 kB/s)
Uploading to gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/maven-metadata.xml
Uploaded to gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/maven-metadata.xml (283 B at 455 B/s)

enter image description here

The problem appears now when I try to use that dependency in another gitlab project in the same group using the same ci_settings.xml which was used by the deployment. I have setup the pom like that:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>de.group.my</groupId>
        <artifactId>parent</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>

    <artifactId>app</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <dependencies>
        ...
        
        <dependency>
            <groupId>de.group.my</groupId>
            <artifactId>model</artifactId>
            <version>0.0.1-SNAPSHOT</version>
        </dependency>
        
        ...
    </dependencies>

    <repositories>
        <repository>
            <id>gitlab-maven</id>
            <url>https://gitlab.my.de/api/v4/projects/255/packages/maven</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
    </repositories>

    <distributionManagement>
        <repository>
            <id>gitlab-maven</id>
            <url>https://gitlab.my.de/api/v4/projects/255/packages/maven</url>
        </repository>

        <snapshotRepository>
            <id>gitlab-maven</id>
            <url>https://gitlab.my.de/api/v4/projects/255/packages/maven</url>
        </snapshotRepository>
    </distributionManagement>

</project>

But when I the run the CI pipeline It always ends up with:

[ERROR] Failed to execute goal on project app: Could not resolve dependencies for project de.group.my:app:jar:0.0.1-SNAPSHOT: Could not find artifact de.group.my:model:jar:0.0.1-SNAPSHOT in gitlab-maven (https://gitlab.my.de/api/v4/projects/255/packages/maven) -> [Help 1]
...
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact de.group.my:model:jar:0.0.1-SNAPSHOT in gitlab-maven (https://gitlab.fronetic.de/api/v4/projects/255/packages/maven

That happens for

But copying the link for the metadata.xml into the browser it is getting downlaoded:

<?xml version="1.0" encoding="UTF-8"?>
<metadata modelVersion="1.1.0">
  <groupId>de.group.my</groupId>
  <artifactId>model</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <versioning>
    <snapshot>
      <timestamp>20211216.121731</timestamp>
      <buildNumber>6</buildNumber>
    </snapshot>
    <lastUpdated>20211216121731</lastUpdated>
    <snapshotVersions>
      <snapshotVersion>
        <extension>jar</extension>
        <value>0.0.1-20211216.121731-6</value>
        <updated>20211216121731</updated>
      </snapshotVersion>
      <snapshotVersion>
        <extension>pom</extension>
        <value>0.0.1-20211216.121731-6</value>
        <updated>20211216121731</updated>
      </snapshotVersion>
    </snapshotVersions>
  </versioning>
</metadata>

In the logs it tries to download:

Downloading from gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/de/group/my/model/0.0.1-SNAPSHOT/model-0.0.1-SNAPSHOT.jar

But this artifact does not exist. It should download model-0.0.1-20211216.121731-6 instead but it doesnt.

Other dependencies can get downloaded (mirrored)

[INFO] Downloading from gitlab-maven: https://gitlab.my.de/api/v4/projects/255/packages/maven/io/micrometer/micrometer-core/1.5.1/micrometer-core-1.5.1.pom
[DEBUG] Writing tracking file /root/.m2/repository/io/micrometer/micrometer-core/1.5.1/micrometer-core-1.5.1.pom.lastUpdated
[DEBUG] Using transporter WagonTransporter with priority -1.0 for https://repo.maven.apache.org/maven2
[DEBUG] Using connector BasicRepositoryConnector with priority 0.0 for https://repo.maven.apache.org/maven2
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/io/micrometer/micrometer-core/1.5.1/micrometer-core-1.5.1.pom
[INFO] Downloaded from central: https://repo.maven.apache.org/maven2/io/micrometer/micrometer-core/1.5.1/micrometer-core-1.5.1.pom (8.2 kB at 91 kB/s)

Upvotes: 1

Views: 3556

Answers (3)

racerm3
racerm3

Reputation: 11

Cause: CI_JOB_TOKEN by default does not have access to the another project's Package Registry

Solution: {Project} > Settings > CI\CD > Token Access

  1. Ensure "Allow access to this project with a CI_JOB_TOKEN" is enabled
  2. Under "Allow CI job tokens from the following projects to access this project"

Enter dependent project path: {username}/{dependent project}

Example: jdoe/hello

Upvotes: 1

Sanjay Bharwani
Sanjay Bharwani

Reputation: 4759

This problem baffled me and whole team for a while before we understood the root cause and the resolution.

So, we had 2 repos in GitLab within same subgroup say SubGroup1. Repo1 refers to a dependency published to Package Registry of Repo2 in the build.gradle file as shown below

repositories {
    mavenLocal()
    mavenCentral()
    maven {
        url 'https://gitlab.example.com/api/v4/projects/**projectID**/packages/maven'
        credentials(HttpHeaderCredentials) {
            name = project.findProperty("gitlab.token.name") ?: "Job-Token"
            value = project.findProperty("gitlab.token.key") ?: System.getenv("CI_JOB_TOKEN")
        }
        authentication {
            header(HttpHeaderAuthentication)
        }
    }
}

Now the problem was happening as for few user's push build was failing with the error

> Could not resolve all files for configuration ':compileClasspath'.
   > Could not find com.example.group:artifact-name:versionname.
     Required by:
         project :

Apparently when few other users simply re-trigger the failed job it worked like a charm, but re-trigger won't help the users who pushed the changes. So it was looking like some magic happening somewhere.

Root cause of the problem was understood from the GitLab Token Documentation. https://docs.gitlab.com/ee/ci/jobs/ci_job_token.html

Below is the summary of the issue While troubleshooting CI/CD job token authentication issues, be aware that: When the CI/CD job token limit is enabled, and the job token is being used to access a different project:

  1. The user that executes the job must be a member of the project that is being accessed.
  2. The user must have the permissions to perform the action.
  3. The target project must be allowlisted for the job token scope limit.

And while troubleshooting the members for which the build was failing, we found they were added in the Repo1 as a Direct Member and were not part of the Repo2.

So ideal solution is, to add them as member in the SubGroup1 so both Repo1 and Repo2 can be accessed as they are within same Group and access is inherited from the Group access. This will also be helpful if new repo added to the Group, members will inherit the access automatically so less maintenance.

Upvotes: 2

VonC
VonC

Reputation: 1324278

After discussion, the OP Mulgard confirms:

I solved it locally.
I needed to add my gitlab user to the project as a maintainer. Adding it to the group was not enough.

But thats only for local.

The gitlab build still does not run since I cant get the CI_JOB_TOKEN inside the docker.
But that is a different topic I guess.

From the project members permissions, at least Developer is needed for package registry publication.

The CI_JOB_TOKEN problem looks like this issue:

We are having this exact same issue on our gitlab-ce environment, with any attempt to push with "CI_JOB_TOKEN" as part of a pipeline resulting in:

 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) 
 on project <our project>: 

 Failed to deploy artifacts: Could not find artifact com.<our namespace>.spring.ext:<our jar>:0.0.1-20190207.112431-1 
 in gitlab-maven (https://git.<our tld>/api/v4/projects/19/packages/maven) -> [Help 1]

I suspect the issues are related as attempting to run the same commands from via the API directly return a 404 for unauthorised users (See: issue 42586).

Modifying our settings.xml to use a personal token with only the "api" scope enabled works.

Upvotes: 1

Related Questions