How to use git lfs in AWS CodeBuild?

Question

Since AWS CodeBuild doesn't seem to support git LFS (Large File System) I tried to install it:

version: 0.2

phases:
  install:
    commands:
      - apt-get install -y bash curl
      - curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash
      - apt-get install -y git-lfs
  pre_build:
    commands:
      - echo Downloading LFS files
      - git lfs pull
  build:
    commands:
      - echo Build started on `date`
  post_build:
    commands:
      - echo Build completed on `date`

For the above code I'm getting the following error (renamed repo address):

[Container] 2020/06/18 16:02:17 Running command git lfs pull
fatal: could not read Password for 'https://username@bitbucket.org': No such device or address
batch response: Git credentials for https://username@bitbucket.org/company/repo.git not found.
error: failed to fetch some objects from 'https://username@bitbucket.org/company/repo.git/info/lfs'

[Container] 2020/06/18 16:02:17 Command did not exit successfully git lfs pull exit status 2
[Container] 2020/06/18 16:02:17 Phase complete: PRE_BUILD State: FAILED
[Container] 2020/06/18 16:02:17 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: git lfs pull. Reason: exit status 2

---

Can I do something else in order to fetch LFS files?

Answer 1

CodeBuild doesn't support Git LFS out of the box. One workaround would be to install it manually, but this will work only if you are connecting to GitHub, BitBucket or other provider directly (e.g. via SSH key).

If you are using it with CodePipeline and using repositories connection (aka "CodeStar Source Connections") then it won't work. When you connect your BitBucket or GitHub account this way, it creates some kind of "proxy" that doesn't support git-lfs resources:

batch response: Repository or object not found: https://codestar-connections.eu-central-1.amazonaws.com/git-http/[..].git/info/lfs/objects/batch

Check that it exists and that you have proper access to it

Failed to fetch some objects from 'https://codestar-connections.eu-central-1.amazonaws.com/git-http/[..].git/info/lfs'

With GitHub however there is a workaround:

GitHub CodeBuild git-lfs workaround

First you have to make sure that in the pipeline's Source stage, source output artifact it set to CODE_ZIP which equals to following setting in the Console:

Then in GitHub, in the repository settings, make sure that git-lfs resouces are included in source code archives:

This will make it work. Now source code downloaded by CodePipeline and passed to CodeBuild will include git-lfs files.

Answer 2

CodeBuild does not support Git LFS, however it's possible to install it on-the-fly and then run git lfs pull from the source directory to download the files. Like this:

env:
  git-credential-helper: yes

phases:
  install:
    commands:
      - cd /tmp/
      - curl -OJL https://github.com/git-lfs/git-lfs/releases/download/v2.13.2/git-lfs-linux-amd64-v2.13.2.tar.gz
      - tar xzf git-lfs-linux-amd64-v2.13.2.tar.gz
      - ./install.sh
      - cd $CODEBUILD_SRC_DIR

  pre_build:
    commands:
      - git lfs pull

<rest of your buildspec.yml file>

Answer 3

CodeBuild does not natively support git LFS. The workaround would be to set up Git LFS 1 and cloning the repository 2 as part of the buildspec.yml execution.

Use 'git-credential-helper: yes' in buildspec for CodeBuild to provide the credentials to git commands 3.