Git not fetching ignored tracked files

Question

This question is almost the opposite of the FAQ of how to stop Git from tracking ignored files. In this case, I want to track them, and not ignore them.

I have a .gitignore file which has the line:

**/*.exe

Most of the time, .exe files are products of builds, and are to be ignored (This project is hosted on a combination of MinGW & Cygwin, so the binaries run on Windows.)

However, there are exceptions to this ignore rule. I have added a few .exe files to my source tree. As I understand it, .gitignore does not apply to tracked files (files in the index).

However, when I clone my respository, the .exe files are not fetched.

If I do a git status on the file in the original repository where I added it, I get:

$ git status program.exe On branch master Your branch is up-to-date with 'origin/master'. nothing to commit, working tree clean

The file also exists in the working tree:

$ ls -l program.exe -rwxrwx---+ 1 abc xyz 23040 Mar 20 15:35 program.exe

$ git log program.exe

commit 412fd58b5a11bf6a40f661109e71f2c0026c1643 Author: abc <abc@xyz.com> Date: Sat Apr 1 02:21:40 2017 -0700

cleanup

commit b0a2efd4dc17b70d046c1e7d78e3142cf29410ba Author: abc <abc@xyz.com> Date: Mon Mar 20 18:41:00 2017 -0700

Initial version

I have pushed the commits to a bare repository on another server. I can find the blob corresponding to the latest checkin in the bare repository:

MyProject.git/objects/41/2fd58b5a11bf6a40f661109e71f2c0026c1643

So obviously it is being tracked. But when I clone the repository, it does not appear in the working tree. On a cloned repository:

$ git status program.exe On branch master Your branch is up-to-date with 'origin/master'. nothing to commit, working tree clean

$ ls -l program.exe ls: cannot access 'program.exe': No such file or directory

I am not using assume-unchanged or the like.

1. Are ignored but tracked files still ignored when fetched in other repositories?

2. How do you force them to be fetched even though they match .gitignore?

3. Do you need to exclude them from matching .gitignore in order to be fetched?

$ git --version git version 2.12.0.windows.1

Edit: Remember, I'm not trying to ignore tracked files after adding them to .gitignore; I'm trying to fetch tracked and committed files, despite them matching .gitignore.

Answer 1

TL;DR: I think there's a key insight here about what tracked means

Between my original answer and a lot of discussion below, I realized something that I think is non-obvious. It certainly was not obvious to me when I first used Git (probably for months if not years). There are two parts here:

• In Git, tracked means in the index.

• But, the index (the next commit you will make) changes every time you check out some specific commit, including git checkout otherbranch, for instance.

This means that whether a file is tracked is a relative thing, not an absolute. File F may be tracked after git checkout branch1 and then untracked after git checkout branch2. It may have been tracked in a past commit on branch master, and untracked now on branch master. If the file is both untracked and ignored, but was tracked in the past and is currently in the work-tree, it can be quite difficult to figure out what's going on. The file will not appear in a new clone, even though it's in the history.

(Original, complete answer below the line here.)

---

However, when I clone my respository, the .exe files are not fetched.

A git clone operation does not fetch any files. Instead, it fetches commits. Now, it's true that commits contain files, so this might seem like a pointless bit of pedantic quibbling, but it's central to your problem, and in fact to almost all of Git, so it really matters. Git doesn't care about files; Git is all about commits.

The commits themselves form the history. Each commit contains some set of files, but also has a back-pointer to a previous commit: "This is what we have now. This is the hash ID of the commit we had before." We call this the parent of the commit. That parent commit also has a parent of its own. This backwards-looking chain is the history in the repository.

A branch name like master is Git's way of recording the current hash ID for that branch. Git starts here, at the most recent commit, using the branch name; and then Git works backwards if needed, obtaining older (parent) commits and going to still-older (grandparent, great-grandparent, etc) commits as requested.

Running git clone copies the branch name, and its commit, and that commit's parent, and the parent's parent, and so on; and that way you get all the history. Each commit carries with it the files that were stored with that commit, so once you have all the commits, you have not only all the current files, but also all the previously current parent-commit files, and every earlier file all the way back to the very first commit. But all these files are hidden away in a secret, Git-only format that is no good to anything but Git. Still, that's what git clone copies.

Of course, to to make Git actually do anything useful, we need the files in some actually-useful form. So Git adds something to this commit history stuff—but it's not something that gets cloned; and that's where your problem comes in.

What's in a repository

In a useful (i.e., non---bare) repository, we need not only all the commits and various branch names to remember their most-recent hash IDs, we also need a work-tree. The work-tree is where we can work with actual files. These are arranged in a tree structure, with top level files and directories, and more files and sub-directories in the directories, as needed. (Hence the name "work tree": it's a tree of working-form files.)

Git also throws in another thing Git calls the index. The short description of the index is that it's where you build the next commit. If you never make any changes to your own repository, this index just kind of gets in your way, but Git still forces you to know about it. If you do make changes, the index is crucial, because the way to make changes is to modify things in the work-tree, then use git add to copy those modifications back into the index, so that they are staged for that next commit. When you then run git commit to make a new commit, Git copies whatever is in the index now—i.e., all the old stuff that was there before, except for whatever you replaced by git adding new stuff—into the new commit.

This really is very important to know: the index tracks what's in the work-tree. In fact, that's exactly what "tracked" means: tracked means in the index. But the index is also what will be in the next commit, and it therefore starts out matching what was in the current commit.

You can—and usually do—have, in the work-tree, files that are deliberately not tracked. For instance, your *.exe files are untracked, on purpose. Being untracked means they are not in the index. That means they are not in the current commit either,¹ and now we can see that there is going to be a problem.

---

¹That is, they are not in the current commit unless the index is "dirty" and thus needs to be written out to a new commit. Again, this is how you make new commits in the first place: you modify stuff in the work-tree—such as modifying files, or adding all-new files, or removing existing files—and then you copy those "dirty" work-tree files into the index, "dirtying" the index. Then you git commit to make a new commit. The new commit's parent is what was the current commit, and its files are whatever was in the index. Git then writes the new commit's hash ID into the branch name, and the new commit is now part of the history, to be saved forever.

---

Objection: "I can find the blob"

You mentioned that:

I can find the blob corresponding to the latest checkin in the bare repository: MyProject.git/objects/41/2fd58b5a11bf6a40f661109e71f2c0026c1643

That means the file was committed at some point. It's in some historical commit. That does not mean that the file is in the most recent commit on a branch like master. The repository has, in Git-only internal (compressed object) format, every version of every file ever committed. And, you can, and often do, have, in your work-tree, a file that is not in the index and not in the current commit. That file might well match a file that you did commit in the past.

Hence, the presence of the file in the repository does not mean it's in the latest commit on some particular branch. It does mean you can extract the historical file. For instance, given the above, you can:

git show 412fd58b5a11bf6a40f661109e71f2c0026c1643 > foo.exe

to extract it to foo.exe at any time. If you know the commit hash and path, you can use that as well:

git show 1234567:path/to/foo.exe > foo.exe

and if the commit has a name (such as a branch or tag name):

git show name:path/to/foo.exe > foo.exe

You can use this method to extract historical files, but it's obviously somewhat painful.

What git clone does after it fetches all the commits

As we noted above, git clone first copies commits and branch names. In fact, by default, it renames these branch-names to what Git calls "remote-tracking branches". But then, as its last step before it quits and calls everything good and ready-to-go, git clone fills in your work-tree.

The way it fills in your work-tree is to git checkout some branch name, usually master. This branch name remembers the hash ID of the latest commit for that branch. That commit was made from an index, and that index did not have .exe files in it. So the most recent master commit does not have .exe files in it.

Since this work-tree is all new, and the commit being checked out has no .exe files in it, your new work-tree has no .exe files in it. That is why they are missing.

If those .exe files do exist in some other commit in the repository, then they are there in the repository, and you can get them out. You just don't get them out with a simple:

git checkout master

because that only extracts, into the index and then on into the work-tree, the files that were saved for the latest commit on master.

So where do you get your .exe files?

Well, that part is up to you. Git gives you no pre-packaged solution here. They may be in the work-tree of the other Git repository you are cloning, but there is nothing in the clone protocol to take the work-tree of the other Git.

One way to deal with this would be to have a commit that contains nothing but the latest .exe files. You could then tell Git:

Extract into my work-tree, without extracting into the index, all the files from the commit whose hash ID is feede3e

for instance, assuming of course that that commit has that hash ID. This turns out to be surprisingly difficult, since Git really wants to extract files from commits into the index before copying them to the work-tree. Putting them into the index makes them tracked, which is not what you want here. Of course, getting those .exe files into a commit in the first place requires—well, what else: wait for it ... have you guessed it yet? Yes, this requires putting those .exe files into the index!

Git just won't let you have it both ways. Either the files are tracked, so they are in the index, so they do go into commits; or they are untracked, so they are not in the index, so they do not go into commits. Pick one and stick with it. If you like, use git worktree add, or a separate clone of this repository, or a completely separate repository, to create a second repository or work-tree that uses a different branch that does have the .exe files tracked and committed. In fact, you might well want this branch or other repository to track and save only .exe files. Then you can simply copy the .exe files over from there.

Or, of course, you can obtain them from some place that does have them, but by some method other than git clone.

(It is actually possible to do all this without a separate work-tree, using various of Git's plumbing commands and a temporary index, but it's fairly tricky and not how Git was built to work. In any case, note that you will want a name—such as a branch name—that you can remember easily that gets you the latest commit with the latest .exe files. The idea of having a branch name that remembers hash IDs for you, with the commit object saving the files, and also remembering the previous commit object ... well, that's a branch. That's what Git does: it keeps branch names that remember commit hashes that remember files that were saved out of the index. So you really do want a separate work-tree and separate index where you update and commit the files. You don't really want to work outside the Git model after all.)

The .gitignore file is a red herring

What an entry in .gitignore does is two main things:

• It suppresses complaints about untracked files. If foo.exe exists in the work-tree but is not in the index and hence will not be saved in the next git commit, Git whines at you. It keeps telling you, over and over again, "Hey, foo.exe is not going to be committed! Remember to add foo.exe to the index! Don't forget foo.exe, or it won't get committed!" This gets obnoxious, and gets in the way of the complaints you want to see, about files you really did forget. So adding foo.exe or *.exe to .gitignore shuts up this complaint.

• It suppresses foo.exe from being automatically added by git add -a or similar. Rather than manually git adding one file at a time to your index, you can add dirty work-tree files en masse by mentioning a directory name, or using --all, or using shell globs like git add *. If foo.exe is currently untracked and is listed in .gitignore, these won't actually copy foo.exe into the index. Even git add * will skip it, with a complaint; git add . or git add -a will skip it with no complaint.

Note that listing a file name in .gitignore has no effect if the file is tracked. Entries in .gitignore are mostly about preventing the file from becoming tracked, not stopping anything that is already in effect.

Answer 2

your build files are on bin folder . so you must add to your .gitignore just :

bin/*.exe or bin/* 

and you should remove this line from your gitignore file :

**/*.exe

edit: if you want to ignore all exe files except some of them , you can exclude them using "!" at the begin of the line pattern , have a look at this https://stackoverflow.com/a/987162/5410373