Why is git is failing to push to origin

Question

I have a process whereby local changes made in a live environment can be committed and merged to a develop branch (so they are available to developers.)

The process goes something like this - within the live directory (which is a checkout of master)

git fetch
git add -A
git checkout -b hotfix
git commit -m"Hotfix of live changes"
git fetch . hotfix:develop
git push origin develop

But the push yields:

$ git push origin develop
To git@gitlab.[...]:group/repo.git
 ! [rejected]        develop -> develop (non-fast-forward)
error: failed to push some refs to 'git@gitlab.[...]:group/repo.git'
hint: Updates were rejected because a pushed branch tip is behind its remote
hint: counterpart. Check out this branch and integrate the remote changes
hint: (e.g. 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

I do not understand why my local develop is behind the origin develop.

The use of git fetch . hotfix:develop is based on what I understand from this article. (It's important to me that I don't check out develop here.) Merge, update, and pull Git branches without using checkouts

Always more to learn with git... :-)

torek · Accepted Answer

OK, per comment, the actual sequence is:

1   $ git checkout master
2   
3   $ git fetch
    [fetch messages occur here]
4   $ git add -A
5   $ git checkout -b hotfix
    [checkout message occurs here]
6   $ git commit -m"Hotfix of live changes"
    [commit output occurs here]
7   $ git fetch . hotfix:develop
    [more git fetch messages here]
8   $ git push origin develop
    [this step fails]

I added the $ command prompt, and some numbers at the left to identify the various steps. I will also assume that all steps succeed except for step 8.

In step 1, you have your index and work-tree in sync with the tip-most commit of branch master in your repository. But the tip commit of master need not have any relationship to the tip commit of branch develop, and this is going to matter when we get to steps 7 and 8. (When we do get to step 7 I'll add some detail about what we can deduce from its success.)

Moreover, we know nothing of what is in the other Git repository, over on origin. Presumably your master is related, in some way, to their master, perhaps even in sync.

Now, in step 2, multiple different developers muddy-up the work-tree. The index still matches the tip of master but the work-tree differs. As an aside, this step is not good practice, since we have no idea, after the fact, who did what: "multiple developers did things." We will (through Git's mechanism) be able to figure out the grand total "things", but not the who did what part—nor the most important part of all, why they did those things.

In step 3, you have your Git contact the Git that handles the other repository, via the URL stored under the remote name origin. This obtains any new commits and tags they have, and any other data needed to make those commits and tags complete and usable. It updates your Git's origin/* remote-tracking branches, so now your origin/master, origin/hotfix (if they have a hotfix), and origin/develop match their master, hotfix (if they have one), and develop.

Again, the "they" step 3 is the other repository, being handled by the other Git, on the system at the URL stored under the name origin. They are also the one rejecting your git push in step 8, so "what they have" is pretty crucial. You'd think this step 3 is important, and it might be, except, we never refer to these origin/* remote-tracking branches ever again!

Now we go on to step 4, git add -A. This copies all files from the work tree into the index. (We're working in our repository; the "they" in steps 3 and 8 fade back into the background and the only "they" left are the various developers.) We don't know who modified files in step 2, or what their intents were, but we can use Git to compare the updated index against the HEAD commit.

It would make more sense, logically speaking, to swap this step with step 5, although the effect will be exactly the same.

Now in step 5 we create a new branch named hotfix. This new branch points to the same commit that we are already on, i.e., to the tip of branch master, which we have not changed in any way since step 1. This particular form of git checkout does not touch anything in the index or work-tree. It only creates a new branch and puts us on that branch. The new branch points to the same commit hash that master points to, so that if, right now, we run:

$ git rev-parse master; git rev-parse hotfix

we will see the same hash ID twice.¹ Or, if we were to draw the commits as a graph, we might see something like this:

...--o--o--*   <-- master, HEAD->hotfix

where * is the current commit, and the series of os are previous commits on branch master, which are also now on branch hotfix.

We still, as yet, have no idea where our own develop points, much less origin/develop, as that's not in any of our steps.

¹git rev-parse means "parse a revision", i.e., turn it into a commit ID. Every branch or tag name simply identifies one specific commit. Branch names simply have the special property of moving to accommodate new commits, as you add them to the branch. Note that they normally move forward, which is where this idea of "fast forward" comes from.

We now move on to step 6, which makes a new commit. The new commit is, as usual, made from the contents of the index, which we set in step 4 to match the contents of our work-tree. (It would make more sense to swap steps 4 and 5: create the new branch first, then update the index. But as long as both steps succeed it doesn't really matter which order we use.) Since our current branch is now hotfix, we end up with this graph drawing:

...--o--o--o   <-- master
            \
             *   <-- HEAD->hotfix

The current commit is now the new one we just made, which is exclusively on hotfix. The previous commits on master are still on master, and are also on hotfix (commits are on all the branches that point to them, even if indirectly).

Note that the new commit's message is "Hotfix of live changes", which is not terribly useful. We can git show the commit to compare it to the previous (tip-of-master) commit, but that won't tell us anything about who made these changes, much less why. Compare to, e.g., a log message reading:

fix special case hit-counter problem

When viewing a page that fribbles a kravitz, if the weebly
area of the jitterbox has glubiframes, the page hit-counter
increments by 3 instead of 1.  We now account for the kravitz
fribbling so that the second adjustment is -1, so that the
total hit counter adjustment is 1 instead of 3.

Not only does this tell us why git show is showing us a -1 instead of a +1 in something related to kravitzes, it tells us what the problem being fixed was. Should this fix introduce a new problem, we know where to start looking.

We now move to step 7. This runs git fetch again, but instead of connecting to another Git, our own Git now plays both sides of the "internet-phone" connection, copying commits from our own repository to our own repository. No actual copying is needed, so the only thing this does is update references—branch names, in our case—as directed by the refspec argument, which in this case is hotfix:develop.

If step 7 succeeds, that tells us a lot about where our develop was before. This particular git fetch command will only move a branch name in a "fast forward" fashion. This means we can redraw our "before" graph a bit. It must have looked something like this:

...--o--o--o   <-- develop, master
            \
             *   <-- HEAD->hotfix

or:

...--o--o   <-- develop
         \
          o   <-- master
           \
            *   <-- HEAD->hotfix

so that the label develop can simply "slide forward" (and maybe down if necessary) so that after the git fetch step, it points to the same commit as hotfix:

...--o--o--o   <-- master
            \
             *   <-- develop, HEAD->hotfix

If the "before" picture had develop pointing to some commit that was not already on hotfix, we would get the same kind of "rejected - non fast forward" error we are getting in step 8.

It doesn't matter how far back our develop was, just that it was somewhere "behind" hotfix. It was able to move in a purely forward direction (rightwards, in these graph drawings), perhaps very fast so as to skip over many commits, to point to the same commit as hotfix.

Importantly, however, we still don't know about origin/develop.

Now we move on to step 8, which actually fails. Here our Git calls up that same other Git as in step 3. This time, though, our Git sends our commits to them, instead of getting their commits over to us. (Hence push instead of fetch.) The one new commit we have that they don't is the one we added in step 6, so that's the one commit we send them.

Having sent them that commit, we then ask them to set their develop to point to this new commit * that we just sent them.

They say no.

The reason they say no is that for them, this operation is not a fast-forward. Let's go take a guessing look at what they have in their graph, now that we have sent them our commit *. It must look something like this—we can't be sure, without going over to their system and running git log --graph commands on their repo, but we can make a pretty good guess:

             o   <-- develop
            /
...--o--o--o   <-- master
            \
             *   <-- [proposed develop]

If they were to move their develop to point to the new commit * that we sent them, they would lose the commit at the tip of their develop. There might be more than one such commit, but there is definitely at least one. They will have to move their develop backward over these commits, to get to the one I'm assuming here is their master, and then forward to *. If there weren't at least one such commit, they would accept our proposal and make their develop point to our new commit * that we just sent them.

Now, the remaining question is: how do we fix this? There are a lot of options, but the most important one is probably to stop using step 2 entirely :-) because step 6 is really poor practice. But if you can't do that yet, then you will need to do a real merge or rebase, to combine the new commit (the one still marked *) with the one at the tip of their own develop.

To do that, you will need a new repository, or at the very least, a new work-tree (see the git worktree documentation). Let's say here that you choose to use a new repository. You can either git clone the existing repository, or git push the existing commit to a new branch in some existing repository. For instance, to push the local develop commit to a new branch on origin, replace step 8 with:

git push origin develop:newbranch

(where newbranch is a name not yet in use as a branch or tag name). Assuming my graph drawing of what's in their repository is accurate, this will produce:

             o   <-- develop
            /
...--o--o--o   <-- master
            \
             *   <-- newbranch

and now, over on that other system, you can git checkout the various branches and merge them or rebase some commit(s) or whatever.

We might also note that step 3, in the eight steps above, is not very useful as it is right now. It updates origin/master and origin/develop in the repository on the live server, but then we never do anything with these updates. We might as well just skip it. If we don't develop directly on the live server in the first place—as we shouldn't—we'll just git fetch and git checkout and git merge or git rebase whatever we need on our development machines, build and test it, and then send the result somewhere (probably a test system for testing, before we send it to the final server for real deployment).

Why is git is failing to push to origin

Answers (2)

Related Questions