Git get worktree for every branch in seperate folders - Bash

Question

I need to write a bash script that copies our repository branches onto our local linux web server every hour.

We have a git remote repository (gitolite) with branches named "master" "testing" "feature-featurename" "hotfix-number" each one of these branches is supposed to have its worktree copied to /var/www/html/branchname

First of all: How do I get different worktrees copied into different folders? Second: How can I automate this process if the "feature-" and "hotfix-" branches constantly change names?

This has nothing to do with a git hook, this is supposed to be a script running on a different server which is triggered by a cron job.

Answer 1

How do I get different worktrees copied into different folders?

That is done with git worktree, as illustrated in Mark Adelsberger's answer.

That answer ends with:

You also could use git worktree list --porcelain instead of searching for worktree directories directly - and that may be preferable in odd cases like (again) namespaced branches.

With Git 2.36 (Q2 2022, see at the end):

git worktree list --porcelain -z

Actually, you should, especially with Git 2.31 (Q1 2021): git worktree list^(man) now annotates worktrees as prunable, shows locked and prunable attributes in --porcelain mode, and gained a --verbose option.

See commit 076b444, commit 9b19a58, commit 862c723, commit 47409e7 (27 Jan 2021), and commit eb36135, commit fc0c7d5, commit a29a8b7 (19 Jan 2021) by Rafael Silva (raffs).
^{(Merged by Junio C Hamano -- gitster -- in commit 02fb216, 10 Feb 2021)}

worktree: teach list verbose mode

^{Helped-by: Eric Sunshine
Signed-off-by: Rafael Silva
Reviewed-by: Eric Sunshine}

"git worktree list"^(man) annotates each worktree according to its state such as prunable or locked, however it is not immediately obvious why these worktrees are being annotated.
For prunable worktrees a reason is available that is returned by should_prune_worktree() and for locked worktrees a reason might be available provided by the user via lock command.

Let's teach "git worktree list" a --verbose mode that outputs the reason why the worktrees are being annotated.
The reason is a text that can take virtually any size and appending the text on the default columned format will make it difficult to extend the command with other annotations and not fit nicely on the screen.
In order to address this shortcoming the annotation is then moved to the next line indented followed by the reason If the reason is not available the annotation stays on the same line as the worktree itself.

The output of "git worktree list" with verbose becomes like so:

$ git worktree list --verbose
...
/path/to/locked-no-reason    acb124 [branch-a] locked
/path/to/locked-with-reason  acc125 [branch-b]
    locked: worktree with a locked reason
/path/to/prunable-reason     ace127 [branch-d]
    prunable: gitdir file points to non-existent location
...

git worktree now includes in its man page:

For these annotations, a reason might also be available and this can be seen using the verbose mode. The annotation is then moved to the next line indented followed by the additional information.

$ git worktree list --verbose
/path/to/linked-worktree              abcd1234 [master]
/path/to/locked-worktree-no-reason    abcd5678 (detached HEAD) locked
/path/to/locked-worktree-with-reason  1234abcd (brancha)
locked: working tree path is mounted on a portable device
/path/to/prunable-worktree            5678abc1 (detached HEAD)
prunable: gitdir file points to non-existent location

Note that the annotation is moved to the next line if the additional information is available, otherwise it stays on the same line as the working tree itself.

And:

worktree: teach list to annotate prunable worktree

^{Helped-by: Eric Sunshine
Signed-off-by: Rafael Silva
Reviewed-by: Eric Sunshine}

The "git worktree list"^(man) command shows the absolute path to the worktree, the commit that is checked out, the name of the branch, and a "locked" annotation if the worktree is locked, however, it does not indicate whether the worktree is prunable.

The "prune" command will remove a worktree if it is prunable unless --dry-run option is specified.
This could lead to a worktree being removed without the user realizing before it is too late, in case the user forgets to pass --dry-run for instance.
If the "list" command shows which worktree is prunable, the user could verify before running "git worktree prune"^(man) and hopefully prevents the working tree to be removed accidentally on the worse case scenario.

Let's teach "git worktree list" to show when a worktree is a prunable candidate for both default and porcelain format.

In the default format a "prunable" text is appended:

$ git worktree list
/path/to/main      aba123 [main]
/path/to/linked    123abc [branch-a]
/path/to/prunable  ace127 (detached HEAD) prunable

In the --porcelain format a prunable label is added followed by its reason:

$ git worktree list --porcelain
...
worktree /path/to/prunable
HEAD abc1234abc1234abc1234abc1234abc1234abc12
detached
prunable gitdir file points to non-existent location
...

git worktree now includes in its man page:

branch currently checked out (or "detached HEAD" if none), "locked" if the worktree is locked, "prunable" if the worktree can be pruned by prune command.

git worktree now includes in its man page:

The command also shows annotations for each working tree, according to its state. These annotations are:

• locked, if the working tree is locked.
• prunable, if the working tree can be pruned via git worktree prune.

$ git worktree list
/path/to/linked-worktree    abcd1234 [master]
/path/to/locked-worktreee   acbd5678 (brancha) locked
/path/to/prunable-worktree  5678abc  (detached HEAD) prunable

---

Before Git 2.36 (Q2 2022), "git worktree list --porcelain"^(man) did not c-quote pathnames and lock reasons with unsafe bytes correctly, which is worked around by introducing NUL terminated output format with "-z".

See commit d97eb30 (31 Mar 2022) by Phillip Wood (phillipwood).
^{(Merged by Junio C Hamano -- gitster -- in commit 7c6d8ee, 04 Apr 2022)}

worktree: add -z option for list subcommand

^{Signed-off-by: Phillip Wood}

Add a -z option to be used in conjunction with --porcelain that gives NUL-terminated output.
As 'worktree list --porcelain' does not quote worktree paths this enables it to handle worktree paths that contain newlines.

git worktree now includes in its man page:

It is recommended to combine this with -z. See below for details.

-z

Terminate each line with a NUL rather than a newline when --porcelain is specified with list.

This makes it possible to parse the output when a worktree path contains a newline character.

git worktree now includes in its man page:

The porcelain format has a line per attribute.

If -z is given then the lines are terminated with NUL rather than a newline.

Answer 2

So first you need a list of branches. For scripting purposes, the command best used for this is for-each-ref. Assuming you just want the local branch names, use something like

git for-each-ref refs/heads/* |cut -d\/ -f3

As an aside, a couple things in the above command assume that you don't use branches in "namespaces". If you use branch names like qa/feature-1 - containing / - then that changes a few things. The above command simply becomes

git for-each-ref refs/heads |cut -d\/ -f3-

but the bigger issue is you probably have to think more about how branch names should map to directory names. So for now I'll proceed with the assumption that branch names won't contain /.

You need to process each branch, so

git for-each-ref refs/heads/* |cut -d\/ -f3 |while read branch; do
  # ... will process each branch here
done

Now you can use git worktree to streamline the individual checkouts. (Note that this should be much more efficient that using archive to copy the whole commit content for every branch, then invoking tar to undo the work you didn't want archive to do in the first place.)

To make sure all required work trees are defined

git for-each-ref refs/heads/* |cut -d\/ -f3 |while read branch; do
  if [ ! -d .git/worktrees/$branch ]; then
    git worktree add /var/www/html/$branch $branch
  fi
done

Now one thing about this is that when the branches are moved (i.e. when pushes are received), it puts the work trees "out of sync" so that you appear to have staged the "undoing" of every change the push did. (The protections for the default work tree don't seem to apply.)

But that seems in line with your requirements; the alternative would be to have the directories updated as pushes come in, which you reject in your description of the problem. So your script should, in that case, sync the worktree to the new changes by "un-undoing" them

git for-each-ref refs/heads/* |cut -d\/ -f3 |while read branch; do
  if [ ! -d .git/worktrees/$branch ]; then
    git worktree add /var/www/html/$branch $branch
  fi
  git reset --hard HEAD
done

Of course sometimes branches go away; if you don't want stale worktree metadata you can add a

git worktree prune

You also could use git worktree list --porcelain instead of searching for worktree directories directly - and that may be preferable in odd cases like (again) namespaced branches.

Answer 3

Horrible few-liner:

mkdir -p /var/www/html
git clone --bare user@git-server:/your-repo.git && cd your-repo.git

git for-each-ref --format='%(refname)' refs/heads/ | grep 'branch-pattern' | while read branchRef; do
  branchName=${branchRef#refs/heads/}
  git archive --format=tar --prefix="$branchName/" "$branchRef" | tar -C/var/www/html -x
done

Let's break it down:

1. Make sure the target directory exists. Probably not necessary for you.
   mkdir -p /var/www/html

2. Clone the git repository and enter the directory
   git clone --bare user@git-server:/your-repo.git

3. List branches. This would be run from a cloned directory. Note that git branch is not used, as that can have surprising outputs when used in scripts.
   git for-each-ref --format='%(refname)' refs/heads/

4. Filter for the branches you want. In your case the pattern would probably be something like grep -E "(master)|(testing)|(feature-.*)" etc.
   grep 'branch-pattern'

5. The while statement reads each branch name and assigns it to the branch variable
   

6. Create a branchName variable that is the name of the branch excluding the ref prefix. Note that this is bash-specific.

7. git archive creates a tar archive of the selected branch, prefixing all entries with the branch name. The archive is sent to standard output
   git archive --format=tar --prefix="$branch/" "$branch"

8. Immediately extract the archive to its target location
   tar -C/var/www/html -x