Constantly Merging .hgignore in Mercurial

Question

This is a process question, so there may be more than one right answer. I'll take anything over what I have now, though.

My team has recently switched to using Mercurial (from Subversion) and for the most part, we like the new power. However, there are a few things that are reducing productivity. One of those things is managing the .hgignore file.

In accordance with the established literature and the advice of "some guys on the internet" :), our team is keeping the .hgignore file up-to-date so that hg addremove always does the right thing. Also, missing files that need to be added is the #1 cause of build failures, so it is important that hg st only returns files that genuinely need action.

The problem is that since we always add new ignores to the bottom of the file, it always causes a merge conflict if two people make .hgignore changes. (Most people use the TortoiseHg client, which adds to the end of the file.) The result of this is that about half the time the file is changed, the person changing it has to handle a merge of .hgignore. This feels a lot like having to monkey with the innards of our source control.

It's causing developers to not want to add files to the .hgignore, since they know it will take at minimum several additional minutes to deal with. We have a fairly large project with a fairly large team that is undergoing constant changes. New build artifacts are introduced fairly regularly, so the problem doesn't appear to be going away. The .hgignore isn't really stabilizing much, since the project is changing a lot. (Which itself is a different issue, of course.)

The 'right' thing to do is 'take both sides' almost always. Technically, two people might have modified the exact same prior line with a text editor, but this is highly unlikely. Unlikely enough that even if the 'take both' approach failed, the consequences are minor.

So, I put the question to the community: How can I improve the situation? Is there a process change that would mitigate this? Is there a tool to automatically take both sides? Can I automate the merge somehow? Is there a checkbox that I can check to magically solve the problem :)?

Edit 1: Here's the (obviously redacted) version of my current .hgignore. You can easily observe that there are several different technologies being used. Several portions of the code are in the midst of being transitioned from one tech to another (VB to C# for example). That causes the set of build artifacts to change and the file needs to be updated.

syntax: glob
*.obj
*.tds
*.map
*.il?
*/obj/*
*/lib/*
*/pch/*
Foo/Engine/frezbat/DocFiles.hpp
Foo/Engine/personal_defines.h
Foo/Engine/revision.cpp
Foo/Engine/frobbish/uLinkHlp.hpp
Foo/Engine/dll/XMLData/**.xml
Foo/Engine/dll/*.syslog
Foo/Engine/dll/*.log
Foo/Engine/dll/Scripts
Foo/Engine/dll/Linked Models
Foo/Engine/dll/prv
Foo/Engine/dll/pub
Foo/Engine/dll/failures.txt
Foo/Engine/dll/users.prm
Foo/Engine/tools/SrvIface/**.dll
Foo/Engine/tools/SrvIface/**.exe
*/dll/*.ini
*/dll/*.exe
*/dll/*.dll
*/quux_obj/*
*.dbg
*~
scripts/backupPath.txt
*.local
*.orig
FooDoc/FooDoc/bin/*
FooDoc/FooDoc/FooDoc.suo
FooDoc/FooDoc/FooDoc.vbproj.user
FooDoc/FooDoc_Setup/Release
Foo/Engine/dll/FooObjects.pdb
Foo/Engine/dll/FooObjects.tlb
Foo/Engine/dll/FooObjects.xml
Foo/Engine/dll/InitechDebugTimer.txt
*.dsk
Foo/Engine/dll/Preferences/CustomToolbar.ini
Foo/Engine/Engine.~dsk
Foo/Engine/dll/ApplicationSettings.fooprefs
Foo/Engine/dll/Foo.cgl
Foo/Engine/dll/IniShare.mem
Foo/Engine/baz_obj/*
Foo/Engine/baz_pch/*
*/dll/*.drc
*.~dsk
InitechBaz/InitechBaz/bin/Debug/InitechBaz.vshost.exe.manifest
InitechBaz/InitechBaz/bin/Debug/InitechBaz.vshost.exe.config
InitechBaz/InitechBaz/bin/Debug/InitechBaz.vshost.exe
InitechBaz/InitechBaz/bin/Debug/InitechBaz.exe.config
scripts/TestComplete/Ottertech_Replay/Log/*
scripts/TestComplete/Ottertech_Replay/[*
relre:ReSharper*
relre:_UpgradeReport_Files
glob:*.suo
glob:*.pdb
*.swp
Foo/Engine/FooObjects/FooObjects/bin/x86/
FooDoc/MCtoDAT/MCtoDAT/bin/x86
FooDoc/Deploy
Foo/Engine/installers/initech-build/bin/*
scripts/TestComplete/Users/*
Foo/Engine/dll/MCtoDAT.xml
FooDoc/Deploy/*
scripts/TestComplete/Ottertech_Replay/Log/*
scripts/TestComplete/Ottertech_Replay/[*
*.orig
Foo/Engine/installers/icon-installers-bin/*
Foo/Engine/Quux/Server/*.esp
Foo/Engine/Quux/BrapServer/*.esp
Foo/Engine/installers/bin/*.exe
Foo/Engine/installers/quux/encryptedsql/*.esp
Foo/Engine/tools/tempfile.tmp
*.pch
*.#*
*.#??
Foo/Engine/dll/frabbing.lib
re:^.*\.\#[0-9][0-9]$
Foo/Engine/Foo/FooObjects/FooObjects/FooObjects.xml
FooDoc/MCtoDAT/MCtoDAT/MCtoDAT.xml
*.sln.cache
FooDoc/TestFooObj/TestFooObj/bin/
*.user
Foo/Engine/FooObjects/FooObjects/FooObjects.xml
Foo/Engine/FooObjects/FooObjects/FooObjects.xml
FooDoc/MCtoDAT/MCtoDAT/MCtoDAT.xml
*/Debug Installer/*.dll
*/Debug Installer/*.tlb
*/Debug Installer/*.xml
*/Debug Installer/*.msi
*/Debug Installer/*.exe
FooDoc/FooDoc_Setup/Debug/*
re:(?i).*\/UpgradeLog.xml
*.sln.cache

Answer 1

Like any tool, Mercurial can only go so far. As pointed out before, most project structures tend to stabilize after a while, making your problems less relevant in a standard use case than in your own.

In your case, it may well be that you just have to live with this.

One alternative would be to restructure your project such that all your ignorable files are co-located, or otherwise referentiable by a wildcard pattern. I would tend to advise against it if the only force in the picture is the need to mitigate ignore management issues. In other words, peculiarities or limitations of your source control system should not constitute a force in deciding project structure.

That said, there are typically genuine forces working to produce a stable project structure, forces that go beyond source control. For instance, the developers need to be familiar with the project, and constant changes impair familiarity. Apparently not the case for you, as you have source code being ignored.

I know this will not go far towards answering your question, but I feel the problem lies outside source control and that in your project those other forces have ceased to exert influence.

Answer 2

Move build-root outside of repo-tree and get sources for builder as result of hg export. This way you can kill two birds with one stone

• build objects would not litter in the repository
• you can verify that all needed files are in repository

Answer 3

Usually there's a flurry of changes in ignore files at the beginning of a project, but it settles in pretty quickly. I'm guessing you aren't using wildcards effectively. If you can't think how to use wildcards in your situation, you probably need to architect your code so newly created ignored files all go into one directory. For example, putting all the build artifacts into a "build" directory. That's a little more up front work in your build scripts, but also helps with things like tab completion since there are only source files in your source directories.