CODEWITHSUNDEEP
gitdockergit-submodulesgitlab-cigitlab-ci-runner
Michał Szydłowski
Michał Szydłowski

Reputation: 3409

How to integrate a project with submodules with Gitlab CI docker runner?

I have quite an unusual use case regarding Gitlab CI and I fail to find a solution that would satisfy my needs.

I have a Gitlab repository that uses a submodule shared by other repositories. I would like to implement a runner that will build a Docker container image for this application. For this purpose I have configured my ci as follows (using docker-in-docker approach):

image: docker:stable

variables:
  # When using dind service we need to instruct docker, to talk with the
  # daemon started inside of the service. The daemon is available with
  # a network connection instead of the default /var/run/docker.sock socket.
  #
  # The 'docker' hostname is the alias of the service container as described at
  # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services
  #
  # Note that if you're using Kubernetes executor, the variable should be set to
  # tcp://localhost:2375 because of how Kubernetes executor connects services
  # to the job container
  DOCKER_HOST: tcp://docker:2375/
  # When using dind, it's wise to use the overlayfs driver for
  # improved performance.
  DOCKER_DRIVER: overlay2

  GIT_SUBMODULE_STRATEGY: recursive

services:
  - docker:dind


before_script:
  - git submodule sync --recursive
  - git submodule update --init --recursive


build_image:
  script:
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
    - docker build -t registry.gitlab.com/proj/name .
    - docker push registry.gitlab.com/proj/name

The problem is that the runner will fail to clone the submodule repository since it has no access to it over SSH, and for HTTPS it would require a username and password which I obviously cannot store in the .gitmodules file that contains the URL from which it's trying to fetch. Not to mention that in this config I have no access to ssh executable in the runner (as well as git so I can't really clone this manually). The error I get is:

Updating/initializing submodules recursively...
Submodule 'src/submodule/my-module' ([email protected]:foo/proj/name.git) registered for path 'src/submodule/my-module'
Cloning into '/builds/proj/name/src/submodule/my-module'...
fatal: cannot run ssh: No such file or directory
fatal: unable to fork
fatal: clone of '[email protected]:foo/prof/name.git' into submodule path '/builds/proj/name/src/submodule/my-module' failed
Failed to clone 'src/submodule/my-module'. Retry scheduled
Cloning into '/builds/proj/name/src/submodule/my-module'...
fatal: cannot run ssh: No such file or directory
fatal: unable to fork
fatal: clone of '[email protected]:foo/proj/name.git' into submodule path '/builds/proj/name/src/submodule/my-module' failed
Failed to clone 'src/submodule/my-module' a second time, aborting
ERROR: Job failed: exit code 1

Needless to say, I cannot afford having this submodule as public repository. I would appreciate any ideas. At this point I'm not even sure if this whole approach is the correct one.

Upvotes: 4

Views: 3865

Answers (1)

Michał Szydłowski
Michał Szydłowski

Reputation: 3409

It turns out it is actually possible to specify a relative path to the other repository in the .gitmodules config file. In this case:

[submodule "src/submodule/my-module"]
    path = src/submodule/my-module
    url = ../my-module.git

That way Gitlab CI will not attempt to clone the repository over HTTPS or SSH like any other, but rather access that path relative to a project it knows it has access to.

Upvotes: 7

Related Questions