CODEWITHSUNDEEP
githubdocker-composegithub-actionsdocker-buildbuilding-github-actions
Ashik
Ashik

Reputation: 3418

How to cache docker-compose build inside github-action

Is there any way to cache docker-compose so that it will not build again and again? here is my action workflow file:

name: Github Action
on:
  push:
    branches:
      - staging
jobs:
  test:
    runs-on: ubuntu-18.04

    steps:
      - uses: actions/checkout@v1

      - name: Bootstrap app on Ubuntu
        uses: actions/setup-node@v1
        with:
          node-version: '12'


      - name: Install global packages
        run: npm install -g yarn prisma


      - name: Install project deps
        if: steps.cache-yarn.outputs.cache-hit != 'true'
        run: yarn


      - name: Build docker-compose
        run: docker-compose -f docker-compose.test.prisma.yml up --build -d

I want to cache the docker build step. I have tried using if: steps.cache-docker.outputs.cache-hit != 'true' then only build but didn't work.

Upvotes: 28

Views: 18480

Answers (5)

Teemu Risikko
Teemu Risikko

Reputation: 3265

If you want to cache only the layers, not the whole images

This answer builds on top of what the user byte said and relates to several other questions.

It is suited for situations where you only want to cache the layers, but not push actual images to a registry. I.e. when you want to use cache type=gha (see Docker Documentation), like recommended in many places. I use this style for temporary images like integration testing images that I don't need to store to a registry, but still want to cache.

  build:
    permissions:
      id-token: write
      contents: read

    name: "Prebuild docker images"
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Cache Docker layers
        uses: docker/bake-action@master
        with:
          push: false
          load: true
          files: |-
            docker-compose.yml
            docker-compose-cache.json

docker-compose-cache.json, replace <GH_ORG>, <REPO> and <SERVICE> with your own references. The scope does not strictly need to match any of those, any unique string is fine.

{
    "target": {
      "service-name": {
        "cache-from": [
          "type=gha,scope=<GH_ORG>/<REPO>/<SERVICE>"
        ],
        "cache-to": [
          "type=gha,mode=max,scope=<GH_ORG>/<REPO>/<SERVICE>"
        ],
        "output": [
          "type=docker"
        ]
      },
   }
}

Upvotes: 2

Byte
Byte

Reputation: 687

This question is old, but I found myself trying to solve precisely the same problem. After reading many different answers and spending lots of time, I eventually found a decent solution.

My workflow file now looks like this:

jobs:
  build:
    name: Integration tests
    runs-on: ubuntu-22.04
    # I need "packages: write" to access GHCR.
    # More about permissions here: https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
    permissions: write-all
    steps:
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Login to Docker Registry
        uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: actions/checkout@v2

      - name: Build docker-compose
        # Before the cache, it was "run: docker compose build".
        run: docker buildx bake --file docker-compose.yml --file docker-compose-cache.json

...

And in the docker-compose-cache.json file, I have the following:

{
  "target": {
    "service-name": {
      "cache-from": [
        "type=registry,ref=ghcr.io/MY_GITHUB_ORG_NAME/service-name:cache"
      ],
      "cache-to": [
        "type=registry,ref=ghcr.io/MY_GITHUB_ORG_NAME/service-name:cache"
      ],
      "output": [
        "type=docker"
      ]
    }
  }
}

For each service in docker-compose.yml, I add a target in docker-compose-cache.json. docker buildx bake takes build instructions from docker-compose.yml and cache instructions from docker-compose-cache.json.

This way, I can still use docker-compose up --build locally as usual.

As you may note, I'm using the GitHub container registry instead of the GitHub actions cache because GHCR has no limitation for cache size.

Upvotes: 21

rdmolony
rdmolony

Reputation: 781

If using docker/bake-action or docker/build-push-action & want to access a cached image in subsequent steps -

  • Use load:true to save the image
  • Use the same image name as the cached image across steps in order to skip rebuilds.

Example:

...
        name: Build and push
        uses: docker/bake-action@master
        with:
          push: false
          load: true
        set: |
            web.cache-from=type=gha
            web.cache-to=type=gha
      -
        name: Test via compose
        command: docker compose run web tests
...

services:
  web:
    build:
      context: .
    image: username/imagename
    command: echo "Test run successful!"

See the docker team's responses;

Upvotes: 3

Vaal
Vaal

Reputation: 684

For those arriving here via Google, this now "supported". Or at least it is working: https://github.community/t/use-docker-layer-caching-with-docker-compose-build-not-just-docker/156049. The idea is to build the images using docker (and its cache) and then use docker compose to run (up) them.

Upvotes: 12

DannyB
DannyB

Reputation: 14776

What you are referring to is called "docker layer caching", and it is not yet natively supported in GitHub Actions.

This is discussed extensively in several places, like:

As mentioned in the comments, there are some 3rd party actions that provide this functionality (like this one), but for such a core and fundamental feature, I would be cautious with anything that is not officially supported by GitHub itself.

Upvotes: 12

Related Questions