CODEWITHSUNDEEP
dockerazure-devopsazure-pipelines
Josiah L.
Josiah L.

Reputation: 351

Azure pipelines: downloading docker image multiple times

We have a single azure pipelines yml that a pipelines that runs many different jobs for testing microservices. We have a single base docker container to make dependency loading much faster.

The problem is that this image gets downloaded for every stage. This defeats some of the point of using a docker container if we cant just reuse the image for different stages. Many times loading image might take 1-5 minutes per job/stage.

How can we use the same docker image throughout different stages without having to redownload it?

name: $(Date:yyyyMMdd)$(Rev:.r)
resources:
  repositories:
  - repository: self
  containers:
  - container: azure_pipelines_deploy2
    image: azure_pipelines_deploy2:1.0.0
    endpoint: acr_connection
trigger:
  tags:
    include: [feature/*]
pr: 
  autoCancel: True # indicates whether additional pushes to a PR should cancel in-progress runs for the same PR. Defaults to true
  branches:
    include: [ master ]

stages: 
- stage: OneOfManyStages

Upvotes: 2

Views: 1842

Answers (1)

Krzysztof Madej
Krzysztof Madej

Reputation: 40899

It is because jobs run on seprate agents, thus your image is not available on that machine. Please check doc here.

A stage contains one or more jobs. Each job runs on an agent. A job represents an execution boundary of a set of steps. All of the steps run together on the same agent. For example, you might build two configurations - x86 and x64. In this case, you have one build stage and two jobs.

You need cache an image on the agent which is not possible at the moment on hosted agent.

So to speed up this you have two option for consideration:

  • create a step (ideally as template reused across your jobs/stages), to build a docker image using already cached image as a base for your image - it could theoretically speed up the process, but if building your image is time consuming it could end up even in worse result - this simply needs to be checked for your scenario
  • use hosted agent and then you will keep your image in the cache, avoiding getting image each time

Upvotes: 3

Related Questions