concourse, how to pull a git repo with all remote branches

Question

I want to create a concourse job that can audit the branches of a git repo. However the git resource and many others like it that I have tried only can pull one branch into a repo. I need to be able to have concourse download a repo with all the remote branches. How can I do that?

I have already tried the following options that do not download all the remote branches:

https://github.com/vito/git-branches-resource

https://github.com/cloudfoundry-community/git-multibranch-resource

I also cannot pull down the extra branches with git commands after the resource is downloaded:

root@e17c8b62-a8ac-4572-5e8f-d880c815ddff:/tmp/build/6bbb901a/my-repo# git fetch
root@e17c8b62-a8ac-4572-5e8f-d880c815ddff:/tmp/build/6bbb901a/my-repo# git branch -r
  origin/HEAD -> origin/master
  origin/master
root@e17c8b62-a8ac-4572-5e8f-d880c815ddff:/tmp/build/6bbb901a/my-repo# git fetch --all
Fetching origin
root@e17c8b62-a8ac-4572-5e8f-d880c815ddff:/tmp/build/6bbb901a/my-repo# git branch -r
  origin/HEAD -> origin/master
  origin/master

fyi this is the info I'm looking for:

$ git branch -r
  origin/HEAD -> origin/master
  origin/asdf
  origin/master
  origin/test

I cannot run extra pull commands as I want to as I want to run this against private repos and do not want to insert my ssh key into concourse.

I did notice that the .git/config in the repos pulled were set to a single branch, meaning that it was configured for remote tracking:

[origin]
fetch = +refs/heads/master:refs/remotes/origin/master

Replacing the master in that line with * solves the issue, but I would like a less manual solution:

fetch = +refs/heads/*:refs/remotes/origin/*

One option is to fix it with a sed command mid job:

cat .git/config | sed -e  's/master/*/g' > .git/config
git fetch --all

git branch -r
  origin/HEAD -> origin/master
  origin/asdf
  origin/master
  origin/test

However I would like something less messy.

Answer 1

This is the correct fetch command that adds in all the remote branches:

git fetch origin '+refs/heads/*:refs/remotes/origin/*'

Answer 2

Just copy/paste the code below inside a task.yml file

---
platform: linux
image_resource:
  type: docker-image
  source: {repository: alpine/git}
run:
  path: sh
  args:
  - -exc
  - |
    git clone https://github.com/octocat/Hello-World.git
    cd Hello-World
    git branch -r

Execute with:

fly -t local execute -c task.yml

You can skip the "git clone" step if you pass the repo as an input.

You can keep this on a task file or move this configuration to your pipeline when you have the expected behavior.

Answer 3

Before you use git fetch you need to do several steps:

• Pass a valid git private key to the container and place it into ~/.ssh/id_rsa

• Add the github host fingerprint to your ~/.ssh/config & chmod 0600

• Set github global config git config user.email "${GIT_USER_EMAIL}" git config user.name "${GIT_USER_NAME}"

Now you can do whatever you desire with git inside your docker container.

Answer 4

Just do git fetch, it automatically does so.