CODEWITHSUNDEEP
cachinggitlabgitlab-cigitlab-ci-runner
jonatzin
jonatzin

Reputation: 972

GitLab CI cache with multiple paths seems to skip a path

I configuring a gitlab CI where I have 2 jobs in the install stage pulling in dependencies into cached locations. Then a job in a different stage tries to access these locations but only one seems to exist.

I've built the CI according to the python example provided by Gitlab, which can be [found here].1

My .gitlab-ci.yml file looks like this.

---

cache:
  paths:
  - foo-1
  - foo-2

stages:
- install
- test

install_foo-1_dependencies:
  stage: install
  script:
  - pull foo-1 dependencies

install_foo-2_dependencies:
  stage: install
  script:
  - pull foo-2 dependencies
  tags:
  - ansible-f5-runner

test_dependencies:
  stage: test
  script: 
  - ls foo-1
  - ls foo-2

The output of install_foo-1_dependencies and install_foo-2_dependencies clearly shows the cache being created. However when you look at the output of test_dependencies it seems only foo-1 cache is being created.

install_foo-1_dependencies output:

Fetching changes...
Removing foo-1/
Checking cache for default-5...
Successfully extracted cache
Creating cache default-5...
....
foo-1: found 1000 matching files                     
Created cache

install_foo-2_dependencies output:

Fetching changes...
Removing installed-roles/
Checking cache for default-5...
Successfully extracted cache
Creating cache default-5...  
....                 
foo-2: found 1000 matching files        
Created cache

Output for test_dependencies

Fetching changes...
Removing foo-1/
Checking cache for default-5...
....
Successfully extracted cache
$ ls foo-1
files
$ ls foo-2
ls: cannot access foo-2: No such file or directory

Upvotes: 2

Views: 1882

Answers (1)

Gerrat
Gerrat

Reputation: 29690

You need to ensure the same runner is used for each stage of this pipeline. From the docs:

Tip: Using the same Runner for your pipeline, is the most simple and efficient way to cache files in one stage or pipeline, and pass this cache to subsequent stages or pipelines in a guaranteed manner.

It's not apparent from your .gitlab-ci.yml file that you're ensuring the same runner picks up each stage. Again from these docs, to ensure that one runner is used, you should use one or a mix of the following:

  • Tag your Runners and use the tag on jobs that share their cache.
  • Use sticky Runners that will be only available to a particular project.
  • Use a key that fits your workflow (e.g., different caches on each branch).

Upvotes: 2

Related Questions