CODEWITHSUNDEEP
gitlabcontinuous-integrationgitlab-cigitlab-ci-runner
kataba
kataba

Reputation: 170

Why use separate jobs and stages on a single GitLab runner?

I have a GitLab pipeline config and a single GitLab runner. There are currently 3 stages in the pipeline:

  1. install_dependencies
  2. lint
  3. test

There is no deployment stage.

The stages for installing dependencies and testing have both 1 job in them. However, the lint stage has 5 linters: ESLint, Prettier, tsc (the TypeScript compiler for type checking), cspell (for spell checking) and jscpd for code duplication checking. The issue is that each of the jobs, although running in parallel, has an overhead of provisioning a Docker container, fetching the git repository, downloading and uploading cache and installing Node packages from cache.

The stage for installing dependencies only runs npm ci by making use of a cached .npm directory.

The test job in the test stage runs Jest.

All of the tasks take little time and can be run in less than 2 minutes even sequentially.

I am curious if there are any benefits of such a pipeline setup except the easy overview of the pipeline progress that the GitLab UI provides? Why would I need multiple stages and jobs if there are no dependencies between jobs and stages in my case and everything may just go under one single job in one single stage?

Upvotes: 3

Views: 1659

Answers (1)

Marcus Ilgner
Marcus Ilgner

Reputation: 7231

One word: parallelism. Especially if there are no dependencies between jobs, the whole pipeline finishes a lot quicker if several of these jobs can run in parallel.

At least that's the idea. Node.js and its millions of small .js files are really hard on Gitlab performance, so in that case you might effectively skip out on that aspect.

You can try having separate package.json files and caches that only install specific linter packages for these jobs.

Upvotes: 1

Related Questions