Pushing git subtree changes to upstream repository fails

Question

I'm trying to ensure that git's subtrees will work for me before I incorporate it into my project. I encountered a problem when pushing subtree changes to the upstream repository.

The setup is that I have two repos, sub and main, and that main includes the sub repo as a subtree.

Then I do the following:

1. Initialize both repos with an initial commit.
2. Update the sub repo directly (i.e. outside of main).
3. Update the sub repo from within the main repo.
4. Split the changes to sub (using git subtree split) into a separate branch, which I then checkout.
5. Attempt to push upstream to the sub repo. Naturally, this push is rejected because it would lose the direct update to sub.
6. Pull the new change from the sub repo.
7. Attempt to push upstream to the sub repo. This time, it should work, but it doesn't.

I've written a script that encapsulates this problem. I'm using git version 1.8.2.1 with the subtree module enabled. Here's the script:

#!/bin/bash

echo -n "Wiping old repositories..."
rm -rf main sub sub-home
echo "done"

echo -n "Initializing main and sub repositories..."
mkdir sub-home
( cd sub-home ; git init -q --bare )
git clone sub-home sub > /dev/null 2>&1
( cd sub ; echo subfile > subfile ; git add subfile ;
  git commit -qm "adding root-level file to sub-project" ;
  git push -q origin master )
mkdir main
( cd main ; git init -q ; echo file > file ; git add file ;
  git commit -qm "adding root-level file to main-project" )
echo "done"

echo -n "Adding sub project as a subtree into main project..."
WD=$PWD
( cd main ; git remote add sub-remote file://$WD/sub-home ;
  git subtree add -P sub sub-remote master >/dev/null 2>&1 )
echo "done"

echo -n "Committing to sub-project directly..."
( cd sub ; date > the-date ; git add the-date ;
  git commit -qm "adding the-date to sub-project"
  git push -q origin master )
echo "done"

echo -n "Committing to sub-project from within main project..."
( cd main ; echo 'subfile what?' > sub/subfile ; git add sub/subfile ;
  git commit -qm "changing sub-project from within the main project" )
echo "done"

cd main
git subtree split -q -P sub -b split-branch >/dev/null
git checkout -q split-branch
echo -e "\nPushing from main subtree to sub project, which should fail:"
git push sub-remote master
echo -e "\nBut if we pull first..."
git pull -q --no-edit sub-remote master
echo "...then a push *should* work (but it doesn't):"
git push sub-remote master
cd ..

And here's the output:

$ ./test.sh
Wiping old repositories...done
Initializing main and sub repositories...done
Adding sub project as a subtree into main project...done
Committing to sub-project directly...done
Committing to sub-project from within main project...done

Pushing from main subtree to sub project, which should fail:
To file:///tmp/git/sub-home
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'file:///tmp/git/sub-home'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first merge the remote changes (e.g.,
hint: 'git pull') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

But if we pull first...
...then a push *should* work (but it doesn't):
To file:///tmp/git/sub-home
 ! [rejected]        master -> master (non-fast-forward)
error: failed to push some refs to 'file:///tmp/git/sub-home'
hint: Updates were rejected because a pushed branch tip is behind its remote
hint: counterpart. Check out this branch and merge the remote changes
hint: (e.g. 'git pull') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

Further git pull commands (from the split-branch branch of main) simply say "Already up-to-date".

The thing that's really confusing to me is that, as far as I can tell, the git push command should really be giving the upstream repo a fast-forward commit, as demonstrated by the following git log output:

$ ( cd main ; git log )
commit 357fe9fb42f5d122338940eb4f22d3ca9d276318
Merge: 472904f cb5d1d3
Author: Jeff Terrell <jeff.terrell@acm.org>
Date:   Fri Apr 19 16:03:03 2013 -0400

    Merge branch 'master' of file:///tmp/git/sub-home into split-branch

commit 472904f432c3a0a89acde02691b8281ac5246fd1
Author: Jeff Terrell <jeff.terrell@acm.org>
Date:   Fri Apr 19 16:03:02 2013 -0400

    changing sub-project from within the main project

commit cb5d1d34ce56374f78c98c5b3f3daa314907b62d
Author: Jeff Terrell <jeff.terrell@acm.org>
Date:   Fri Apr 19 16:03:02 2013 -0400

    adding the-date to sub-project

commit 7d1942203d30e0d9e8663517e6d594545bc50640
Author: Jeff Terrell <jeff.terrell@acm.org>
Date:   Fri Apr 19 16:03:02 2013 -0400

    adding root-level file to sub-project
$ (cd sub ; git log )
commit cb5d1d34ce56374f78c98c5b3f3daa314907b62d
Author: Jeff Terrell <jeff.terrell@acm.org>
Date:   Fri Apr 19 16:03:02 2013 -0400

    adding the-date to sub-project

commit 7d1942203d30e0d9e8663517e6d594545bc50640
Author: Jeff Terrell <jeff.terrell@acm.org>
Date:   Fri Apr 19 16:03:02 2013 -0400

    adding root-level file to sub-project

Here are my questions (finally):

1. Why is this push rejected?
2. What can I do about it? (If using the --force option is the answer, how can I be certain I am not doing something destructive?)
3. Is there a better way to use the subtree module to avoid this problem? (Note: I'm not willing to use submodules.)

Answer 1

Your problem is not really related to git subtree. You having problems with the good old tricky git ui. In this case git push. You obviously assumed that its syntax follows git pull. That was rather naive – you are using git ;).

You push output tells you what is wrong here:

To file:///tmp/git/sub-home
! [rejected]        master -> master (non-fast-forward)

git pull sub-remote master fetches and merges the head of sub-remote/master into your currently checked-out branch, just as you expected. But git push sub-remote master does not push the head of your checked-out branch into sub-remote/master. It does push the branch of the same name. So in this case that is master, as you can see in the output above.

From git help push (syntax is git push <repsitory> <refspec>):

The format of a <refspec> parameter is an optional plus +, followed by the source ref <src>, followed by a colon :, followed by the destination ref <dst>. It is used to specify with what <src> object the <dst> ref in the remote repository is to be updated.

The <dst> tells which ref on the remote side is updated with this push. Arbitrary expressions cannot be used here, an actual ref must be named. If :<dst> is omitted, the same ref as <src> will be updated.

So the command you are looking for is git push sub-remote splitbranch:master. But why are you not using git subtree push in the first place?