CODEWITHSUNDEEP
node.jsgulpnpm-installgitlab-cigitlab-pages
user341554
user341554

Reputation: 599

GitLab CI: How can I reuse installed npm packages between jobs?

I have a GitLab Pages site that uses Gulp for building. My .gitlab-ci.yml file looks similar to this:

image: node:latest

before_script:
  - npm install gulp-cli -g
  - npm install gulp [...and a whole bunch of packages] --save-dev

build:
  stage: build
  script:
  - gulp buildsite
  artifacts:
    paths:
    - public

pages:
  stage: deploy
  script:
  - gulp
  artifacts:
    paths:
    - public

cache:
  paths:
  - node_modules/

Before both the build and pages jobs, the npm install commands are executed (once before each job). Since I have quite a few packages, this usually takes a while.

Is there a way to only do the installs once across the entire build?

I thought that's what cache was supposed to help with, but it still seems like it still redownloads everything.

Upvotes: 26

Views: 14860

Answers (3)

Dane Rossenrode
Dane Rossenrode

Reputation: 212

There are 2 ways to do it; using cache and using artifacts. You can read this article about the differences, but essentially:

  • If your job does not rely on the the previous one (i.e. can produce it by itself but if content already exists the job will run faster), then use cache.
  • If your job does rely on the output of the previous one (i.e. cannot produce it by itself), then use artifacts and dependencies.

Here is a simple sentence to remember if you struggle between choosing cache or artifact:

→ Cache is here to speed up your job but it may not exist, so don't rely on it.

So in this case, if you don't need to access the node_modules folder after the pipeline is complete, then use cache. Simply add this to the stage that contains the build command:

cache:
  untracked: true
  paths:
      - node_modules/

Then make sure to run the same build command in the later stage - but it will run much faster (nearly instantly) since the node_modules directory will already exist with the latest versions of all dependencies.

If for some reason you want to access (download) the node_modules folder after the build is complete, use artifacts. Add this to the build step:

artifacts:
  paths:
    - node_modules

And add this to the following steps that need the node_modules directory:

artifacts:
  paths:
    - public

Upvotes: 0

lk_vc
lk_vc

Reputation: 1182

You need to set cache: untracked: true to really cache the files not tracked by Git.

cache:
  untracked: true
  paths:
      - node_modules/

Upvotes: 4

Clive Makamara
Clive Makamara

Reputation: 3045

Though the answer in the comments is essentially correct. I think a specific answer for your case would be good though. One approach you could use is adding a third stage that would bear the load of installing node modules plus you could also cache them to speed up subsequent builds:

image: node:latest

stages:
  - prep
  - build
  - deploy  

before_script:
  - npm install gulp-cli -g  

prep:
  stage: prep
  script:
  - npm install gulp [...and a whole bunch of packages] --save-dev
  artifacts:
   paths:
   - node_modules 
  cache:
   paths:
   - node_modules

build:
  stage: build
  script:
  - gulp buildsite
  artifacts:
    paths:
    - public

pages:
  stage: deploy
  script:
  - gulp
  artifacts:
    paths:
    - public

This solution will only perform the installation once and will cache the result for future ci pipelines, you may also place an expiry on the node modules artifacts.

Upvotes: 11

Related Questions