git pre-push hook: run test on each new commit

Question

Context

I want to ensure that each commit I push pass tests.

I want to check this on my (client) side, i.e. before commits are even pushed (so I don't want to rely on CI tools).

Problem

Currently, I have implemented a pre-commit hook that run my tests, so that I cannot even commit a broken state.

However, my test suite takes more than a few seconds to run. It is that much time I need to wait prior to writing my commit message. This makes it impractical to use on a daily basis; both because I frequently commit, and that I sometimes purposefully want to commit a broken state to be squashed later (I know about git commit --no-verify, but that is not the point).

Question

So instead of checking each commit one at a time (at creation), I want to batch-test them before pushing.

How to implement a pre-push hook that run my test suite for each new commit to be pushed?

(For the sake of simplicity, say that passing tests means test/run_tests.sh returns 0.)

Answer 1

Thanks to phd's hint (in comments) and a shameless pillage of git's own example, I have drafted the following ./.git/hooks/pre-push hook (that I took care to chmod +x beforehand).

It seems to do the job in vanilla situation, we'll see how it goes over time. Anyway, improvements are welcome!

#!/usr/bin/sh

# An example hook script to verify that each commit that is about to be pushed
# pass the `./run_tests` suite. Called by "git push" after it has checked the
# remote status, but before anything has been pushed.
# If the test suite (and so the script) exits with a non-zero status, nothing
# will be pushed.
#
# In any case, we revert to the pre `$ git push` state.


# Retrieve arguments
remote="$1"
url="$2"

z40=0000000000000000000000000000000000000000 # SHA of a non existing commit


# Save current "git state"
current_branch=$(git rev-parse --abbrev-ref HEAD)

STASH_NAME="pre-push-$(date +%s)"
git stash save -q --keep-index $STASH_NAME


# Do wonders
while read local_ref local_sha remote_ref remote_sha
do
        if [ "$local_sha" = $z40 ]
        then
                # Handle delete
                continue # to the next branch
        elif [ "$remote_sha" = $z40 ]
        then
                # New branch, examine all commits
                range="$local_sha"
        else
                # Update to existing branch, examine new commits
                range="$remote_sha..$local_sha"
        fi

        # Retrieve list of commit in "chronological" order
        commits=$(git rev-list --reverse $range)

        # Loop over each commit
        for commit in $commits
        do
            git checkout $commit

            # Run the tests
            ./test/run_tests.sh

            # Retrieve exit code
            is_test_passed=$?

            # Stop iterating if error
            if [ $is_test_passed -ne 0 ]
            then
                echo -e "Aborting push: Test failed for commit $commit,"\
                  "with following error trace:\n"
                # something like: tail test/run_tests.log
                break 2
            fi
        done
done


# Revert to pre-push state
git checkout $current_branch

STASH_NUM=$(git stash list | grep $STASH_NAME | sed -re 's/stash@\{(.*)\}.*/\1/')
if [ -n "$STASH_NUM" ]
then
    git stash pop -q stash@{$STASH_NUM}
fi
#removed fi

# Return exit code
exit $is_test_passed

Answer 2

from ebosi's answer i added some modifications that might interest some of you:

1. Each time i iterate the loop of commits i recompile the test binary (the right version of the test for the right commit). This is done in the shell function "recompile_test" (note: you might want to double check that the variable "is_test_passed" defined in the scope of this function is still valid in the scope of the whole script itself)
2. if a test succeeds i display a nice "OK message", plus a reminder of the commit message: echo "`git log -1 --oneline $commit`\n"
3. if a test fails, i dont break and exit! i display a nice "KO message", plus a reminder of the commit message. and a summary of the trace of the tests just like ebosi did in his answer.
4. This way at the very end. if the very last commit passed the tests i still can push.
5. also when done debugging i added a couple of "> /dev/null" here and there
6. also had to modify the sed instruction, so that it works on mac os x (catalina version 10.15.7, sed is a built-in)

# An example hook script to verify that each commit that is about to be pushed
# pass the `./run_tests` suite. Called by "git push" after it has checked the
# remote status, but before anything has been pushed.
# If the test suite (and so the script) exits with a non-zero status, nothing
# will be pushed.
#
# In any case, we revert to the pre `$ git push` state.


# Retrieve arguments
remote="$1"
url="$2"

z40=0000000000000000000000000000000000000000 # SHA of a non existing commit


# Save current "git state"
current_branch=$(git rev-parse --abbrev-ref HEAD)

STASH_NAME="pre-push-$(date +%s)"
git stash save -q --keep-index $STASH_NAME

recompile_test()
{
   echo "\n\033[32mRecompiling tests suite...\033[0m"
   make test 2>&1 > /dev/null
   # Retrieve exit code
   is_test_passed=$?

   if [ $is_test_passed -ne 0 ]
   then
       echo "\033[31m[\033[mKO\033[31m] Aborting push: Tests failed for commit $commit\n"\
           "\t\033[31mCOULD NOT COMPILE TEST SUITE\n\033[0m"
       break
   fi
}

# Do wonders
while read local_ref local_sha remote_ref remote_sha
do
       if [ "$local_sha" = $z40 ]
       then
               # Handle delete
               continue # to the next branch
       elif [ "$remote_sha" = $z40 ]
       then
               # New branch, examine all commits
               range="$local_sha"
       else
               # Update to existing branch, examine new commits
               range="$remote_sha..$local_sha"
       fi

       # Retrieve list of commit in "chronological" order
       commits=$(git rev-list --reverse $range)

       # Loop over each commit
       for commit in $commits
       do
           git checkout $commit > /dev/null 2>&1

           recompile_test;

           echo "\n\033[32mRunning tests...\033[0m"
           # Run the tests
           ./tester 2>&1 > ./tests_logs/last_test_output.log

           # Retrieve exit code
           is_test_passed=$?

           # Stop iterating if error
           if [ $is_test_passed -ne 0 ]
           then
               echo "\033[31m[\033[mKO\033[31m]\033[0m Aborting push: \033[31mTests failed for commit $commit\n\033[0m"
               echo "commit brief:"
               echo "\033[38;5;160m`git log -1 --oneline $commit`\033[0m\n"
               echo "\t\033[31mERROR TRACE:\n\033[38;5;132m"
               cat ./tests_logs/last_test_output.log
               echo "\033[0m"
           else
               echo "\033[32m[\033[0mOK\033[32m] \033[0mTrying to push: \033[32mTests Passed for commit $commit\n\033[0m"
               echo "commit brief:"
               echo "`git log -1 --oneline $commit`\n"
           fi
       done
done

#added so it doesnt prevent us to checkout... we rm if the file exists
[ -f tests_logs/output.xml ] && rm tests_logs/output.xml
# Revert to pre-push state
git checkout $current_branch 2>&1 > /dev/null

STASH_NUM=$(git stash list | grep $STASH_NAME | sed -e 's/stash@{\(.*\)}.*/\1/')
if [ -n "$STASH_NUM" ]
then
   git stash pop -q stash@{$STASH_NUM}
fi

if [ $is_test_passed -ne 0 ]
then
   echo "\033[31m[\033[0mKO\033[31m] \033[0mPUSH ABORTED (failed on last commit)\n"
else
   echo "\033[32m[\033[0mOK\033[32m] \033[0mPUSH POSSIBLE (final commit successful)\n"
fi
# Return exit code
exit $is_test_passed```