git Error: cannot update the ref 'refs/heads/master'

Question

I'm trying to sync two working systems together. I'm very new to git and I think I messed up by editing and committing on the two W10 systems onnected over a VPN via mapped drive letters differently which diverged,I understand I shouldn't have done that. But my main data is on System2, so I deleted my System1 directory completely, then did a clone to it. The --no-hardlinks is necessary to make the clone work with Windows mapped drives.
git clone --no-hardlinks c:/cap2office s:/cap2office
So everything I would think would be in sync.

On System1 (the target of the clone) I do git status and it says everything up to date, nothing to commit.

On System2, I edited one file, added and committed, and I do:

PS C:\cap2office> git status
On branch master
Your branch is ahead of 'origin/master' by 2 commits.
(use "git push" to publish your local commits)
nothing to commit, working tree clean

Where did the commits come from? One I can understand because I edited a file but where did the other come from? And before I edited that one file, it did the same thing, but said I had 1 commit for some reason.

Elsewhere on stackoverflow somebody suggested on System1:
git pull origin -r master
but that didn't help

So when I try a push from System2 to System1 (as the error suggests):

PS C:\cap2office> git push s:/cap2office [should push the repository to my laptop 'remote']
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 456 bytes | 456.00 KiB/s, done.
Total 4 (delta 2), reused 0 (delta 0), pack-reused 0
remote: Checking connectivity: 4, done.
remote: error: refusing to update checked out branch: refs/heads/master
remote: error: By default, updating the current branch in a non-bare repository
remote: is denied, because it will make the index and work tree inconsistent
remote: with what you pushed, and will require 'git reset --hard' to match
remote: the work tree to HEAD.
remote:
remote: You can set the 'receive.denyCurrentBranch' configuration variable
remote: to 'ignore' or 'warn' in the remote repository to allow pushing into
remote: its current branch; however, this is not recommended unless you
remote: arranged to update its work tree to match what you pushed in some
remote: other way.
remote:
remote: To squelch this message and still keep the default behaviour, set
remote: 'receive.denyCurrentBranch' configuration variable to 'refuse'.
To s:/cap2office
! [remote rejected] master -> master (branch is currently checked out)
error: failed to push some refs to 's:/cap2office'
PS C:\cap2office>

I don't understand what it's telling me and how to fix it. I never checked out or in anything AFAIK.

And in the git-scm tutorial talks about two people collaborating, but I think it leaves some steps out, they never mention checkout or checkin, just file modification, so maybe that lack precipitated my confusion and this mess.
+++++++++++++++++++EDIT+++++++++++++++++++
After studying torek's excellent answer (call me if you're in Albuquerque for a dinner on me :-) which did move my knowledge along, my main stumbling block has to do with merging the remote with the local. I've read git chapter 3 several times now but am still missing something because I can't get the remote version merged to get everything in sync on my local machine.

On my laptop I do: git status and it says "your branch is ahead of 'origin/master' by 1 commit.
git checkout remotes/origin/master
(It says I am in "detached HEAD" state)
(says Previous HEAD position was e6b... Had to recommit a few files after reset to get things in sync again)
git diff master remotes/origin/master (shows that I want the remotes/origin/master version)
git merge master (to use the remote version as the latest - what I want - merge local master into it)
(it should do a commit to make my local master working files be the remote version)
(it should also move 'master' to point to the updated version received from the remote?)
(or how do I make the local 'master' point to the version updated from the remote?)
git checkout master (to make my local copy updated from the remote to be my working copy)
(Switched to branch 'master' (or how do I make my 'master' be after the merge?)
(Still says "your branch is ahead of 'origin/master' by 1 commit.")
(It didn't update my working copy from the remote version.)
(It looks like it didn't update local 'master' to point to the remote version of the files)
++++++++++++++++++EDIT++++++++++++++++++
I may have found the magical incantation, but I don't fully understand it.
I did:
git fetch (what did it fetch?)
git rebase origin/master (did that move my local master pointer to the fetch?)
then git status didn't indicate I was ahead of origin/master anymore.

Answer 1

This:

 remote: error: refusing to update checked out branch: refs/heads/master

is perfectly normal; it means everything is working correctly.

I never checked out or in anything AFAIK.

That's not really relevant.

The issue here is simple enough in a way, but getting to understanding it is a bit complicated. We want to start with this: Git is a distributed version control system in which everyone has every commit. That is, there are many copies of the repository floating around.

Now, not all copies can all be up-to-date at the same time. Some version control systems handle this problem by having a dedicated "source of truth" repository. Everyone might have "everything", but whoever is the SSOT has the real copies. That's a straightforward way to handle things, but it's not Git's way.

The Git repository

Instead, in Git, every repository is its own private kingdom (or fiefdom or whatever you want to call it). Each Git repository believes its version of truth. Whatever you have in your Git repository, that's the "right stuff".

Besides this, we start with one other point: Git is all about commits. Git is not about files (though we store files in commits), and not really about "branches" either, although there are a lot of different things in GIt that people call "branches", usually without saying which of the many thing(s) they mean. (So be careful with the word branch: you might not take from it what whoever said it meant. They might not even know what they meant, or said!) The key items in any Git repository are the commits.

The commits are numbered, but—unfortunately for humans—with what look like random garbage. Other version control systems number the commits too, but in an easier way: e.g., in SVN, you make commit #1, then the next commit is commit #2, and so on. SVN can do this because in SVN there's only one repository. There aren't any other repository copies at all (other than pure cache ones anyway). To make a new commit, you have to go back to the (single) SVN and have it allocate the next number.

In Git, your repository is yours. You make new commits whenever you want! Nobody else has them—yet. They're yours. They still get a unique number, and to make that happen, Git allocates enormous numbers (between 1 and, currently, 2¹⁶⁰-1; future versions, being experimented with now, have an even bigger space) using a trick—literally just a cryptographic hash—that makes sure that (a) no two commits ever get the same number and (b) every Git repository agrees that your new commit's number, whatever it is, is the right number for that commit. (This trick literally cannot work forever, and won't, but the sheer size of the hash IDs mean it won't fail for billions of years, we hope.)

To avoid making humans memorize a bunch of random-looking numbers, a Git repository stores names too. These are branch names, tag names, and all kinds of other names, and each name stores one of these random-looking hash-ID numbers. That's all Git actually needs, due to more clever trickery.

A Git repository, then, consists principally of these two databases:

• One—usually far bigger—database holds commits and other supporting objects. These objects have random-looking hash ID numbers. The commit IDs in particular are unique, not just to this one database, but across the entire universe of all Git databases. Git finds the objects in this database by their hash IDs. You must have the hash ID to find the object!

• The other database, usually much smaller, holds names. Each name maps to one hash ID. When the name is a branch name, the hash ID the branch name holds is, by definition, the last commit in the branch. Humans will generally ask Git for a commit by branch name: Git will look up the branch name in this database, find the latest commit hash ID, and thus be able to get the latest commit.

This is what you need to know about a repository, that it consists mainly of these two databases. You make new commits by having Git store a commit (and its supporting objects) in the big database, and update the hash ID in the little database so that the branch name now records the new commit. The new commit itself records the old commit.

The commit

Because the repository is all about the commits, you also need to know what a commit is and does for you. The details can get much more complicated if you want, but this too is actually pretty simple. We already know that each commit is numbered, and this number is how Git finds the commit. The number is a cryptographic checksum of all the stuff in the commit. This makes the commit itself completely read-only. That is, we can, if we like, extract a commit from the database, make changes, and stuff the result back into the database—but if and when we do this, we get a new commit with a different number. The old commit is still in there. In a way, this becomes a sort of append-only database.

Each commit has two parts:

• A commit holds (indirectly) a full snapshot of every file. These are the files you will be able to use, if you have Git extract the commit. They're stored in a compressed and Git-only format, with their contents de-duplicated. This means that, because we re-use files all the time from one commit to the next, the full-snapshot aspect doesn't cause the storage requirements to explode: the files are only logically in the commit. They're physically shared.

• Separately, each commit holds some metadata, or information about the commit itself. This includes, e.g., the name and email address of the person who made the commit (e.g., yourself), recorded forever (no part of any commit can ever be changed), or as long as the commit itself continues to exist, and copied out of your user.name and user.email setting at the time you make the commit.

To make one of the things we call "branches" work, every commit stores, in its metadata, a list of the raw hash IDs of previous commits. There's usually exactly one hash ID in this list. Git calls this (or these) the parent (or parents) of the commit. It's why a branch name can just hold the hash ID of the latest commit: the latest commit itself holds, in its metadata, the hash ID of the second-from-latest commit, which in turn holds the hash ID of the third-from-latest, and so on, all the way back to the very first commit.

Pictorially, the branch name "points to" the last commit, which "points backwards" to its parent, which points backwards to its parent, and so on. The act of making a new commit just means:

• freezing a new snapshot for all time;
• writing out some metadata, including the current commit's hash ID as the parent;
• turning all of this into a new commit; and
• writing the new commit's new unique hash ID into the branch name.

Because Git keeps the next commit prepared at all times (with help from you), this git commit action goes fast.

Connecting two repositories with git fetch

Once we make new commits, we have commits that some other repository doesn't. We therefore need to connect two repositories to each other now and then. When we do that, we pick one repository as the sender, and one as the receiver.

If we think they have made new commits and we want to obtain their new commits, we use git fetch. This has our Git software call up their Git software, using some URL or path name. Our Git has them list out their latest commit hash IDs, as found by their branch names (which we have them list as well). Our Git software checks to see if we have these commits. Since the hash IDs are unique, all we need to find out is: do we have those hash IDs? If we do, we have those commits. If not, we ask them for those commits. Asking for a commit implies that they must tell us about that commit's parent commit hash IDs, which we can then ask for as well.

Once we see commit hash IDs that we do have, we don't need those commits because we already have them. In this way, we ask for exactly those commits that they have, that we don't. Their Git software packages up those commits (and the files that go with them, with more cleverness here) and sends us the objects we need. Our Git adds those commits and other objects to our objects database.

Once we have their commits, we need a way for our human to find their commits. So our Git takes each of their branch names and renames those. You might wonder why this renaming happens. It's the first key issue here, but we'll get back to it in a moment. Our Git renames their branch names, so that they become our remote-tracking names.

We need to take a slight sidebar here:

git pull origin ...

I always recommend people avoid git pull to start with. The pull command is a convenience command; it means:

1. run git fetch; then
2. run a second Git command of my choice

and things get a lot more complicated with the second command (in part because it's a choice, and in part because no matter which one you choose, there are complications). It helps newbies a lot to separate out any problems from step 1—there usually aren't any—from any problems that come up with the second command, of which there can be many, but they depend on which second command you chose (and you probably don't even know which one you've chosen yet!).

The origin in git pull origin ... is handed off to git fetch (in fact, most of the arguments to git pull go through to git fetch). Here, origin is the name of a remote. A remote, in Git, is just a short name for "some other Git repository". The specific name origin is the standard short name for "the repository I cloned".

So origin means "the repository from which I got all my initial commits". That repository is, of course, a Git repository, which means it has its own branch names. When you start making commits of your own, you're going to need your branch names.

Some systems (e.g., Mercurial) handle this by just assuming that you'll create your own branch names that will be different from someone else's branch names. This takes the attitude that we're all sharing here. Git isn't like that: Git is much more "I am the Lord of the Manor, these are all mine!" So instead of using their branch names, your Git takes each of their branch names and changes them, inserting—in this case—origin/ in front of each. These are now remote-tracking names.

So, if they have master, you now have origin/master. This is how your Git remembers that they had a master. If you're doing a git fetch origin, your Git software will call up their Git software using the URL remembered under the name origin, and will find out any new commits they have that you don't and bring them over and then update your origin/master: your memory of their master.

So git fetch is pretty straightforward, once you settle into this "I am the ruler of my domain" mindset: you get their branch names, which find their latest commits, and then you get their commits (which have unique numbers) and store them away and remember their branch names as your remote-tracking names instead.

Meanwhile, any branch names in your repository are your branch names, to do with as you will.

Making new commits

We've already taken a quick high-level look at the idea of making new commits, but now it's time to look into the mechanics a bit more. In particular, you mentioned:

PS C:\cap2office> git status
On branch master
Your branch is ahead of 'origin/master' by 2 commits.
(use "git push" to publish your local commits)
nothing to commit, working tree clean

Where did the commits come from? One I can understand because I edited a file but where did the other come from? And before I edited that one file, it did the same thing, but said I had 1 commit for some reason.

You have to have made these commits. When you ran git clone, you got all their commits, and your Git renamed all their branches—git clone actually runs git fetch for this step—and you then had origin/master. Your own Git software then made your master to match their origin/master, and did a checkout of this particular commit. That is, your git clone did a git checkout for you.

You may have run git commit itself only once, but there are several other commands that make commits. However it happened, you (and/or your Git) made two new commits. This is how you got "two ahead".

The usual way to make a new commit is:

• edit some file(s);
• git add; and
• git commit.

There's something very odd here though. Remember I said earlier that no part of any commit can ever be changed. That's true. So how did you edit some files?

The answer is that when Git checked out a commit, it extracted the permanent archive into a work area. This work area—your working tree—contains the current commit.

How does Git know which commit is the current commit? Git needs the hash ID of the commit. What Git saves, though, is the current branch name. When you use git checkout or git switch to select some branch, Git stores the branch name in what I call an "attached HEAD". (Git does not have a specific term for this, but it does have one for "detached" HEAD, when HEAD contains the raw hash ID. So since this is the other case, "attached" is the obvious thing to call it.)

The branch name itself holds the latest commit's hash ID, by definition. So we now have a situation where:

• HEAD holds the name master;
• master holds some random-looking hash ID, which is the current commit;
• the files from that commit are available to you, in your work area.

When you modify these files, in your work area, you're not touching the commit itself: that's stored, frozen for all time (or as long as the commit exists), in the big all-objects database. You're also not touching Git's pre-prepared, ready-to-go next commit. Git has this stored away in something Git uses three names for.

This three-named thing, which Git calls the index, the staging area, and (rarely these days) the cache, is where Git keeps a new snapshot ready to go. When you run git add, you're telling Git to update the proposed snapshot. Git reads the existing working tree files you name, compresses and de-duplicates them, and updates its proposed next commit.

When you run git commit, Git:

• uses the files in the index / staging-area to make the new snapshot;
• adds the appropriate metadata, including the current commit hash ID (which is in the branch name which is in HEAD); and
• turns all that into a new commit, which Git then stuffs into the branch name (in HEAD) in the smaller database

and now your master branch is ahead by another commit.

When you use git merge, git revert, git cherry-pick, and various other Git commands, Git can make additional commits. These, too, make use of the index and your working tree. There are in effect three copies of every "active" file at all times:

• one, frozen forever, is in the current commit;
• one, in the frozen format but replaceable, is in Git's index/staging-area;
• the usable one, an ordinary file in an ordinary folder, is in your working tree

and these copies (or "copies" when they're de-duplicated duplicates) participate in a carefully choreographed dance, where git checkout and git add and git commit and so on know which ones to update when.

Connecting two repositories with git push

First, a side note: to use git push, you should in general use the remote idea as before. That is, git push origin, not git push s:/cap2office. The reason for this is just so that Git can update remote-tracking names if and as needed. It's not strictly necessary. You can git fetch from a raw URL or path as well, but if and when you do that, Git is unable to update remote-tracking names. Git can only build the remote-tracking name if you use the remote, after all.

The git push command is as close as Git gets to the opposite of git fetch, but it has one critical difference. As with git fetch, you should name a remote; Git will use the remote name to look up the URL, and will call up some Git software on the remote. Your Git will now offer, to the remote, some hash ID(s) based on what you give to git push. For instance, if you run:

git push origin master

your Git software looks up the URL for origin, then turns your branch name master into the hash ID of your latest commit.

Your Git now offers their Git that commit, by hash ID. As with git fetch, they check their big database to see if they have it. If they do, they'll say "no thanks, already have that one", but if not, your Git now has to offer its parent by hash ID too. This repeats until they say "already have that one". Your Git now knows which commits to send (and by implication, any new files you might need to send with them, and by further implication from the "already have that one", which files are duplicates and don't need to be sent). So your Git packages up all the objects to send over, so that they can add them to their Borg collective repository.

Once you've sent the objects, though, your Git now asks their Git—politely, assuming you don't use --force, which you generally shouldn't—to please, if it's OK, set one of their branch names to record the the latest of the new commits you just sent from your master. The name your Git is going to ask their Git to set is, of course, master.

In other words, you'd like them to make their name master hold a new "latest commit". There are two ways for this to go wrong:

1. They might have their own later commit, that you're now asking them to drop. This isn't the case here, but you'll encounter it very soon.

2. They might have their master branch checked out. This is the case you are hitting.

Remember that each branch name records exactly one hash ID. They have some commit, with some hash H, checked out right now. Their HEAD holds the name master, and their master holds hash ID H.

You're asking their Git to replace the hash ID in their master name with some new, later commit:

... <-F <-G <-H   <-- master

should become, in their repository:

... <-F <-G <-H <-I <-J   <-- master

just as things are in your repository.

But if they do that, the carefully choregraphed dance with the three "active" copies of each file will get messed up! Their index / staging-area holds the files from commit H right now! Their working tree holds the files from commit H! They know this because their name master holds hash ID H.

If they change the hash ID in their name H, things will go wrong. In particular, suppose whoever's on that computer, typing away right now while working, editing those files, getting ready to git add and git commit, runs git add and git commit while their Git is replacing the hash ID. They don't have the right proposed next commit. They're going to undo your work.

That's what this means:

remote: error: By default, updating the current branch in a non-bare repository
remote: is denied, because it will make the index and work tree inconsistent
remote: with what you pushed ...

You're going to mess up Fred who's actively working right now on that computer.

Now, you know there is no Fred over there actively working. You can go over to that other computer later and fix things up. But Git doesn't know that, and you probably were not even aware that this would be required. (Well, now you are.)

But that's kind of a common-ish situation, so:

remote: You can set the 'receive.denyCurrentBranch' configuration variable
remote: to 'ignore' or 'warn' in the remote repository to allow pushing into
remote: its current branch; however, this is not recommended unless you
remote: arranged to update its work tree to match what you pushed in some
remote: other way.
remote:
remote: To squelch this message and still keep the default behaviour, set
remote: 'receive.denyCurrentBranch' configuration variable to 'refuse'.

You can set things up so that you can do this to yourself, in other words. Or, if you like, you can set things up so that your Git tells you "don't do that" without printing out quite so much noise. To do these things you will want to configure the receive.denyCurrentBranch setting in the Git repository over on the other computer.

These are (usually) the wrong ways to get work done

The danger here is from git push. It interferes with ongoing, but not yet committed, work because it's something you are doing, on your computer, that is making changes to someone else's system. It can cause a "non fast forward" error even if you're pushing to some other branch, because it's something you are doing, on your computer, that is making changes to someone else's system.

That's the common thread to the problem: you're changing someone else's system with your git push. This problem does not occur—can not occur—with git fetch, because with git fetch, you're getting something from someone else—something they've already committed—and putting it in your system.

If you can completely avoid git push, these problems all go away. Just do your work on your computer. Commit it and make it available to someone else—even if that "someone else" is yourself, tomorrow or later the same day—and then fetch someone else's work.

Bare repositories, including hosting sites

There's a third alternative, useful if you can't use an "always fetch" model. The big problem with git push-ing to an active repository is that you could mess with ongoing work that someone is doing in that repository. But what if you could make a repository in which it is physically impossible to do any work? This would be a repository that just sits there holding incoming commits.

This is what a --bare clone is. Making a bare repository gives you a place where people can git push. This still has potential issues with two different users trying to git push different new commits to the same name, but Git has a built in way to reject that, as a "non fast forward" error. Whoever "wins the race" by pushing first gets their commits in; the "loser" then has to git fetch the new commits, do whatever is required to combine work, and then git push in such a way that their new work adds on to the winner's work, rather than discarding the winner's work.

This is what sites like Bitbucket and GitHub and GitLab offer: they give you a shared bare repository, plus a whole bunch of add-on features designed to attract you away from their competitors, to entice you to use their systems. Their value grows in part from the network effect, so they're willing to give you some of this "for free" (of course nothing is truly free, but there may be zero additional monetary cost to you, and the value they add may be very valuable to you, so this can be a win-win situation).

git log

A final note:

Your branch is ahead of 'origin/master' by 2 commits.

Where did the commits come from?

You can find out. The git log command displays commits. It works by the simple principle of starting from some starting-point you name, and working backwards—forever by default, or until some stopping-point you name, if you like. In this case:

git log origin/master..master

tells your Git software to start with the commit named by your master branch-name. Your Git will display a summary of that commit: its hash ID, author information, and log message in particular. Then it will move back one hop, to that commit's parent. It will display a summary of that commit as well. It then will try to move back one more hop, but at this point it will have run out of the two commits that you are "ahead of" origin/master: that remote-tracking name names the commit that your Git remembers seeing in the other repository, and that commit is, presumably, the parent of the parent of the most recent commit in your repository.

This two-dot stop..start syntax (which, like most things in Git, is backwards) generally selects all the commits starting with start, and working backwards until it encounters commits that would be found by starting with stop and working backwards. This distinction becomes useful when we deal with "branches" of this form:

          I--J   <-- branch1
         /
...--G--H
         \
          K--L   <-- branch2

Here, the two branch names pick out commits J and L respectively. The parent of J is I, and the parent of I is H, and so on, backwards; the parent of L is K, and the parent of K is H—not J! So the two "branches" come back together at commit H. In Git, commit H is (currently) on both branches at the same time. So are all the earlier commits (G and further backwards).

The two-dot syntax still works: branch2..branch1 means commits J and I. We're excluding commit L explicitly with the branch2.. part, but that in turn excludes K, which excludes H, which is what causes the backwards walk that starts at J and moves to I and then to H to stop. If it helps you picture how this works, imagine taking two temporary highlighter pens. Put the green one on branch1 and work backwards, and put the red one on branch2 and work backwards. Red overrides green: you can do either first, or do them at the same time as long as red overrides; when you're done marking, whatever's green comes out, whatever is red does not, and anything not marked at all does not either (e.g., if there's some branch3 pointing to the last of some commits we didn't ask for, they don't come out).

This kind of graph is a key to using Git. You need to be able to "think" in graph terms, or draw the graphs, or learn to have Git draw them (with, e.g., git log --graph).

You can also have Git compare snapshots, using git diff or git log -p. The latter has Git walk backwards as usual, but at each step, Git will compare the parent commit's snapshot to the to-be-displayed commit's snapshot. Whatever is different, Git summarizes; whatever is the same, Git doesn't mention at all.