Git pull mentions untracked files that I can't find

Question

I'm trying to pull from Git and get the error:

The following untracked working tree files would be overwritten by merge:

:q
Please move or remove them before you merge.

I can neither find this file nor know what it might be referring to, or why it starts with a :

git diff shows that it as a new file

Answer 1

(Greatly expanded from comment; perhaps an answer, although there is not enough information in the original question to tell.)

First, a side note: Is this on Windows? Windows has "issues" with files with a colon in the name. (I avoid Windows as much as I can, and am not sure about this, but I think it should be impossible even to create such a file there.) Other systems should handle it just fine, and it should not be mysteriously hidden—but I'll explore two possibilities below.

Another side note: git pull just runs git fetch (to get commits from another Git and put them into your repository under origin/master and the like), followed by git merge (to incorporate the commits now in your origin/master into your own master). I did not set up a remote, so below, I use git merge directly, with a named side-branch in my own repository, rather than one of these origin/master things (which are a lot like branches, but are not—at least not quite—actually branches, because Git terminology is terrible).

Finding oddly named files

You say that you can't find the :q file—but (a) don't mention what tools you are using to look for it, and (b) do say that git diff shows a file named :q as a new file (again without quoting actual git diff output, making it hard to guess what you mean). This means you can see the file, if only indirectly, through some tool other than the one you're using that would normally show you files directly.

As an experiment, I made a new repository and created a few dummy files and a side branch within it. In the side branch, I created and committed two files with "interesting" names. One has the four character name colon Q backspace backspace—naive programs could "display" this file's name by writing the :q string to the display, then writing the two backspaces to the display, so that the next characters overwrite the colon and lowercase-q. The next file name could then overwrite the one just displayed. The other "interesting name" file is colon Q space, which on display, looks like :q without a space, since most of the screen's space is filled with, well, space, which looks exactly like spaces. :-)

Now, the ls command on this particular system is not so naive, so running ls we get:

$ ls
:q??    :q      README  file

shown on the screen, though if I run ls -C | expand I get:

$ ls -C | expand
        :q      README  file

As you can see, or perhaps "can't see" would be better phrasing, the file with the backspaces in its name appears to have vanished, except for occupying one slot in the column-ized file listing. We do know it's there though!

Finding oddly named files via git status

If I now run git status, this is what it shows:

$ git status
On branch master
Untracked files:
  (use "git add <file>..." to include in what will be committed)

    ":q\b\b"
    :q 

nothing added to commit but untracked files present (use "git add" to track)

Here, Git has taken care to show the embedded-backspace name specially: it has added double quotes and used \b to encode the backspace. The name that is spelled with a trailing space, however, is still rather cleverly disguised: it shows up as colon, Q, space, and the "space" looks exactly the same as if there were no space.

What Git does when merging

Since I have committed these files on the side branch sidebr, any attempt to merge them produces:

$ git merge sidebr
error: The following untracked working tree files would be overwritten by merge:
    :q??
    :q 
Please move or remove them before you merge.
Aborting

Again, Git has done something clever(ish) with the backspaces, but the trailing blank is simply copied to the screen (well, Terminal window), where it looks the same as the background.

Removing the unwanted files from the work-tree

To remove the unwanted files, I can simply remove all files beginning with colon and q:

$ rm :q*
$ ls
README  file

after which git merge sidebr succeeds:

$ git merge sidebr
<git makes me enter a merge commit message>
Merge made by the 'recursive' strategy.
 ":q\b\b" | 1 +
 :q       | 1 +
 2 files changed, 2 insertions(+)
 create mode 100644 ":q\b\b"
 create mode 100644 :q 

Removing the unwanted files in the next commit

Now the unwanted files are back—after all, I committed them in branch sidebr on purpose, specifically to set up this problem. But now they're no longer untracked files, so to remove them, I want to use git rm rather than plain rm, and then I want to commit the removal as well.

At this point I ran into another issue:

$ git rm :q*
fatal: pathspec ':q??' did not match any files

What is happening here is that Git treats arguments starting with colon specially. To work around the problem, I first removed the files normally:

$ rm :q*

then told Git to update all the files it knows about, which of course includes these two :q<otherstuff> files:

$ git add -u

Now git status shows:

$ git status
On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    deleted:    ":q\b\b"
    deleted:    :q 

and I can run git commit:

$ git commit -m 'remove icky colon-named files'
[master 1e4e9f6] remove icky colon-named files
 2 files changed, 2 deletions(-)
 delete mode 100644 ":q\b\b"
 delete mode 100644 :q 

Since you're getting this same complaint from git merge, once you actually remove the unwanted file from your work-tree, your git pull or git merge will bring it back as a committed (rather than untracked) file and you will need to commit a removal, using something like the two step process above.

For completeness (no need to read this section): why git rm :q* failed

There is an alternative trick, which I include here for illustration. The issue Git has with files named :q<space> and the like is not the space, but the leading colon. This is documented under the definition of pathspec in the Git glossary.

A pathspec that begins with a colon : has special meaning. In the short form, the leading colon : is followed by zero or more "magic signature" letters (which optionally is terminated by another colon :), and the remainder is the pattern to match against the path. The "magic signature" consists of ASCII symbols that are neither alphanumeric, glob, regex special characters nor colon. The optional colon that terminates the "magic signature" can be omitted if the pattern begins with a character that does not belong to "magic signature" symbol set and is not a colon.

In the long form, the leading colon : is followed by a open parenthesis (, a comma-separated list of zero or more "magic words", and a close parentheses ), and the remainder is the pattern to match against the path.

In this particular case, :q<backspace><backspace> was treated as:

• begin using pathspec magic (: starts the magic)
• do no magic (q is alphanumeric, and is not an open parenthesis, so this is the short form and there are no magic characters)
• use the name q<backspace><backspace>

and of course the file is not named q<backspace><backspace> but rather :q<backspace><backspace>.

It would work to type in ::q<backspace><backspace> here, but the point of using :q* in the shell is that typing in backspace characters, without having them just erase in the usual fashion, is difficult, so this defeats the whole point.

Using rm :q* and git add -u makes Git search the directory, which sidesteps the pathspec magic. But there's another simple trick: the file whose name is :q<backspace><backspace> has an infinite variety of names, most of which don't start with a colon. The simplest of these is ./:q<backspace><backspace>. (Others include ././:q<backspace><backspace>, and so on, for an infinite number of leading ./ sequences.) So we just get the shell to generate these names for us:

git rm ./:q*

and now we are ready to commit.

I went with the git add -u method above because it's most likely that whoever created and committed this oddly-named file, whatever its name is—it may be :q<space>, or :q<space><space>, or :q<tab>, or :q<newline>, among the many possibilities—probably used git add . or git add -A or the like to get it added to a commit in the first place.

Answer 2

It looks to me like you type :w:q instead of :wq when trying to exit vi. It's not clear to me why a merge would clobber this file, but if its existence is indeed accidental then it's safe to remove it.

I'm not sure why there would be an issue with finding it. But I can't see why

rm :q

(in the work tree root) wouldn't resolve the issue, if indeed you know this to be a junk file