CODEWITHSUNDEEP
git-submodulesamazon-ecscircleci
baku
baku

Reputation: 775

Circleci: Private git subdmodule with AWS ECR orb

I have the following config.yml for a circleci build which works well it's using the aws-ecr and aws-ecs orbs. 

version: 2.1
    orbs:
      aws-ecr: circleci/[email protected]
      aws-ecs: circleci/[email protected]
    workflows:
      build-deploy:
        jobs:
          - aws-ecr/build_and_push_image:
              account-url: "myaccount.amazonaws.com"
              repo: "my/repo"
              region: us-east-1
              tag: "${CIRCLE_BRANCH}"
              filters:
                branches:
                  only: mybranch

The problem is this repo contains a .gitmodules file which pulls in a private subdmodule. I can't seem to figure out how to override/extend the orb to run additionally circleci equivalent of

git submodule update --init

I have tried adding this to the dockerfile, but then i get 

Permission denied (publickey).

fatal: Could not read from remote repository.

note: the dockerfile builds fine locally, since local docker inject my git key automagically

I tried reconfiguring the orb job into steps also, i.e. 

version: 2.1
orbs:
  aws-ecr: circleci/[email protected]
  aws-ecs: circleci/[email protected]
workflows:
  build-deploy:
    jobs:
      - lb_build_and_push_image:
        steps:
          - add_ssh_keys:
              fingerprints:
                - "my:fin:ger:print"
          - aws-ecr/build_and_push_image:
              account-url: "account.amazonaws.com"
              repo: "my/repo-backend"
              region: us-east-1
              tag: "${CIRCLE_BRANCH}"
              filters:
                branches:
                  only: mybranch

...where fingerprint is from the 'user key' in ssh checkout keys. I've tried various configurations of jobs/steps.

And schema always fails with usual message of:

Error: ERROR IN CONFIG FILE:
[#/workflows/build-deploy/jobs/0] 0 subschemas matched instead of one
1. [#/workflows/build-deploy/jobs/0] expected type: String, found: Mapping

does anyone have pointers on how to proceed, what the right config might be, or just general pointers of how to move forward in troubleshooting? any insight much appreciated.

Upvotes: 2

Views: 1187

Answers (2)

odie5533
odie5533

Reputation: 562

You can tell CircleCI to checkout the submodules using job pre-steps and setting the aws-ecr/build-and-push-image checkout parameter to false. The trick is that we're doing the checkout (and submodule updating) ourselves and then telling the orb to skip that step.

You also need to set a User Key in CircleCI so that CircleCI has access to your private GitHub submodule repositories. Follow the directions in the CircleCI Help Page to add the User Key using a GitHub machine user.

version: 2.1
orbs:
  aws-ecr: circleci/[email protected]
  aws-ecs: circleci/[email protected]
workflows:
  build-deploy:
    jobs:
      - aws-ecr/build-and-push-image:
          filters:
            branches:
              only: mybranch
          pre-steps:
            - checkout # run checkout manually
            - run: # so that we can pull the submodules too
                name: "Pull submodules"
                command: |
                  git submodule sync --recursive
                  git submodule update --recursive --init
          checkout: false # tell the orb job not to run checkout
          repo: "my/repo-backend"
          tag: "${CIRCLE_BRANCH}"

This code is using aws-ecr 7.3.0 because the 8.0.0 and later has a bug which does not like the checkout parameter. Hopefully the bug will get fixed in the future.

Upvotes: 1

baku
baku

Reputation: 775

this was the eventual solution. newer release of aws-ecr orb supplies commands for steps 

version: 2.1
orbs:
  aws-ecr: circleci/[email protected]
  aws-ecs: circleci/[email protected]
  aws-cli: circleci/[email protected]

jobs:
  build_and_push_image:
    docker:
      - image: circleci/python:3.7.1
    steps:
      - checkout
      - run:
          name: "Pull Submodules"
          command: |
            git submodule init
            git submodule update --remote
      - setup_remote_docker
      - aws-ecr/build-image:
          repo: "my/repo"
          tag: "${CIRCLE_BRANCH}"
      - aws-cli/install
      - aws-ecr/ecr-login
      - aws-ecr/push-image:
          repo: "my/repo"
          tag: "${CIRCLE_BRANCH}"

however, this did rely on updates to aws orb, i would be interested if there was another way to solve this, assuming those steps had not been exposed as commands

Upvotes: 1

Related Questions