CODEWITHSUNDEEP
dockergitlabgitlab-ci-runner
Terry
Terry

Reputation: 317

Pipeline built on the top of GitLab is failing because of docker persistent volumes

I am setting up CI/CD pipeline in gitlab. I have self hosted gitlab version 11.4 running on my server. I am using shell executer for gitlab-runner instance. In my source code, there is a docker-compose.yml file which has 8 different containers (mysql, elasticsearch, php, nginx, queue).

When i push new code, it triggers defined pipeline and stopped with following errors

Running with gitlab-runner 11.6.0 (f100a208)
  on iZAPLabs runner 9a639f82
Using Shell executor...
Running on ip-10-250-142-190...
Fetching changes...
warning: could not open directory 'docker/.data/db/ginvoicing_dev/': Permission denied
warning: could not open directory 'docker/.data/db/mysql/': Permission denied
warning: could not open directory 'docker/.data/db/performance_schema/': Permission denied
warning: failed to remove docker/.data/redis/dump.rdb: Permission denied
warning: failed to remove docker/.data/db/ib_logfile0: Permission denied
warning: failed to remove docker/.data/db/ginvoicing_dev: Permission denied
warning: failed to remove docker/.data/db/ibdata1: Permission denied
warning: failed to remove docker/.data/db/auto.cnf: Permission denied
warning: failed to remove docker/.data/db/mysql: Permission denied
warning: failed to remove docker/.data/db/performance_schema: Permission denied
warning: failed to remove docker/.data/db/ib_logfile1: Permission denied
warning: failed to remove docker/.data/elasticsearch: Permission denied
warning: failed to remove docker/var/run/php: Permission denied
ERROR: Job failed: exit status 1

.data directory has docker volumes mounted in the different containers. But i don't know why new push is trying to remove them. I even try to use "cache" directive. But no luck.

Any help would be appreciated.

Upvotes: 2

Views: 2050

Answers (1)

Markus
Markus

Reputation: 3158

I guess the gitlab runner itself is running as user git or gitlab or gitlab-runner or something like that (Just put whoami in the script section to determine if you don't know).
Most of your docker containers are running as root user inside the container. So everytime you use a volume which is located inside your git repository, the permission will change when your containers are changing the data. After your containers were stopped, you gitlab user is not able to clean up the repository for a new job.

Some ideas (but maybe no one will solve your problem)

a) add your gitlab runner user to the docker group b) try to run sudo git reset --hard c) mv the location of your docker volumes to outside your repository, maybe to /tmp (and don't forget to delete them after the job is done).

Upvotes: 1

Related Questions