How can I exclude everything but all top-level dot files and a subdirectory?

Question

In keeping track of my essential configuration files with Git, I'm having trouble writing my .gitignore file.

My goal is to include two things in my repository:

1. All dotfiles in my user directory ~/, which is my gitroot.
2. the hidden folder ~/.vim/bundle

So far, I have tried the following approach, to no avail.

.gitignore:

# Ignore everything
*

# Include top-level dot files.
!/.*

# Include vim plugins
!/.vim/bundle/*

This only stages the dotfiles in my gitroot.

Note.

My gitroot is my user directory ~/.

Update

In the following form, the ~/.vim folder is staged:

# Ignore everything
*

# Include top-level dot files.
!/.*

# Include vim plugins
!/.vim

So, I'm guessing that I should express the pathname /.vim/bundle differently, somehow.

Answer 1

TL;DR

You're running into an optimization that breaks your use case: once Git decides not to look inside a directory, it never finds the files within that directory that it would add. To fix it you need at least one more rule:

!/.vim/bundle/

You can use a different rule to defeat all optimization:

!*/

which should also make the problem go away (at the expense of much deeper searches which result in a lot of slowness).

Long

You have a good start, but you have run into a problematic optimization (that the Git folks are, I think, trying to fix). We must also make assumptions about what is currently in your index (aka staging-area) as the index contents affect how Git scans through directories. Let's assume that your index is truly empty, as it would be right after git init-ing a new repository. That way, since there are no existing index entries, they cannot affect Git's normal directory-scanning process.

Let's suppose your work-tree here has files named README and .profile, and directories d/ and .vim/. Your .gitignore has three rules. The first one is a positive rule ("do ignore") and the second and third are negated rules ("don't ignore"). The rules themselves, minus the !, are:

*
/.*
/.vim/bundle/*

Now when Git is working on README it first checks *, which matches, then /.*, which does not match, then /.vim/bundle/* which also does not match. The last matching rule takes effect: that's * which says "do ignore" so README will be ignored. The file .profile, on the other hand, the first two rules, so the second one applies, and it says ! so .profile is untracked but not ignored: git status will whine about it, and git add . will add it to the index.

Now let's consider how d/ and .vim/ are handled. These are directories, not files: Git is not going to save them, but it may or may not save files that are within them. To do so, it will have to read and check all the file names within them. This is pretty expensive, so Git will first try to skip the directory entirely. This is the problematic optimization I mentioned.

Anyway, d/ matches the first rule but not the other two. This means Git gets to skip reading d/ entirely. git status will not whine about it and git add will not scan it.

We're all good so far. .vim/ matches the second rule (but not the third) so Git will look inside .vim.

Let's say that .vim/ contains .netrwhist, after/, bundle/, and maybe some other directories. .netrwhist matches *, does not match /.*, and does not match /.vim/bundle/*, so the * match applies and it gets ignored. The after/ directory matches *, does not match /.*, and does not match /.vim/bundle/*, so it goes unread.

Now we hit the problem: .vim/bundle/ is a directory. Git checks this path name against your .gitignore directives. bundle/ matches *. .vim/bundle does not match /.* as that means files in the top level; .vim/bundle/ is a directory and is not in the top level. The path also does not match the third directive /.vim/bundle/*, which requires that this be a file or directory within .vim/bundle/. So the only matching directive is the first one, and .vim/bundle goes unread just like d/ did.

This means that Git never finds any of the files or directories within .vim/bundle/. It never checks the files (if any) against anything and it never scans the subdirectories for more files. You need to force Git to look inside .vim/bundle/.

An entry in the index changes a lot

If there is some file in the index whose path name is .vim/bundle/a/b, Git is going to have to scan the directory .vim/bundle/a. So any new files placed in there will show up! I'm not sure if Git ever skips .vim/bundle/ itself in this case (in theory it could but I don't think it does in practice). Once you have a file path in the index, that file is tracked and its presence in any .gitignore becomes irrelevant.

Note, however, that files that are in the index now are not necessarily in the index tomorrow, if you remove them or if you check out some other commit where those files are not present. The whole thing is a little bit tricky.

Answer 2

I was inspired by this answer to write the following adequate .gitignore file. :)

# We use a whitelisting approach to track certain configuration files.
# Apparently, this needs to be done recursively because git can't see past a directory that is not
# included. Hence, if we want to include a subdirectory, we first need to include its upper
# directory, by the whitelisting procedure as shown below.

# Exclude all files and non-hidden directories.
/*

# Include all dotfiles.
!.*

# Exclude all hidden directories.
.*/

# Include the upper directory.
!/.vim

# Exclude all files and non-hidden directories.
/.vim/*

# Include the vim plugins.
!/.vim/bundle

Answer 3

The leading slash indicates an absolute path relative to gitroot; remove it to select all dotfiles.

# Ignore everything
*

# Include all dot files.
!.*

# Include vim plugins
!/.vim/bundle/*

Here is a handy and concise reference with examples.

You will need to stage and then commit .gitignore for the changes to take effect. Happily you can repeatedly git commit --amend with further changes to keep the history clean until you have what you want.

As a vaguely-related aside, I've often seen people attempt to match "a folder of name folder anywhere in the project" with /folder, where it ought to be **/folder.