Hg : join many repositories into a single one

Question

I use mercurial to manage my sources. Today I have 12 differents repositories with about 6 branches for each. Each repository contains a maven project. I would to create a parent project and declare all my projects as a module and merge all the projects into a single repository. To be visual, today I have : -Repo1/project1 -Repo2/project2 ... -Repo12/project12

And I want : - globalRepo with - module1 - module2 - ... - module12

Keeping the branches and the historics (if possible).

What is the best practice to do that ? (Subrepo, merge...)

Thx !

Answer 1

Here is my list of solutions in order of complexity, starting with the simplest.

Forget History

The simplest. Put the current state of your default branch of each project in the in the new project structure. Then start a new repo.

We haven't been told how active the branches are, but if they are active you can do the same for each branch:

hg up null       # not strictly necessary, but may avoid the impression that a branch started now.
hg branch <name>
<populate directory structure with the current state of a branch from each project>
hg commit

You will still have access to the history in the original project repos. What you won't be able to do so easily is compare new changesets with old history. Obviously, the need for this diminishes with time, but it may be important for you.

Merging

Retaining history by merging repos is possible. First, each project repo has to have all it's files moved to a subdirectory to reflect the intended structure of the global repo:

for <each project>
    md projdir
    hg rename * projdir
    hg commit

Then each project can be brought into the global project:

cd globalrepo
hg init
hg pull <first subproject>
for <other subprojects>
    hg pull -f <subproject>     # the -f is necessary because the projects are unrelated.
    hg merge
    hg commit

And this should give you a merge with full history.

When comparing files with 'old' history, remember (or use aliases) to use the --git option for commands that have a --git option (that's diff and export at least). Bear in mind that while history may be complete, it may be confusing because the history of each project will be contiguous.

Using Subrepos

It is possible to join repos using subrepos along the lines you suggest. Having said that, subrepos are intended to be not-too-active and independent components. Neither of these appear to be true in your use case.

In your case, the projects won't be independent because they are only used in the context of the new global repo, and not shared with other global repos.

If the subrepos are active, as they will in your use case, you have to be vigilant in remembering to commit both the subrepo and the parent repo. Failure to remember this can cause irritation because pushes can be aborted due to multiple heads. That will happen if person A pushes a changeset to a subrepo without pushing the parent repo; then person B updates to the tip of the parent repo (which doesn't pull the new subrepo changeset), and attempts to push his commit on the same subrepo. If everyone on your team is competent in Hg then it might well be fine for you, but if not I'd advise against using subrepos for your use case.

If there are active branches, the subrepos will retain them, but the global repo won't see them unless it is also branched:

cd globalrepo
hg up null       # not strictly necessary, but may avoid the impression that a branch started now.
hg branch <branch>
for <each subrepo>
    cd <project dir>
    hg up <branch>
cd globalrepo
hg commit

I hope you get the idea (untested, this bit is from memory).

Answer 2

It is possible to pull unrelated repositories together and merge them. The trick is to rename the files into specific directories before doing the merge.

For example, The following script creates three separate repositories with three files in the root of each project. It then combines them into a single repository where the files from each project are moved to subdirectories:

@REM Create three separate repositories
hg init Repo1
hg init Repo2
hg init Repo3
@REM Add and commit files to each of them.
cd Repo1
echo >file1
echo >file2
echo >file3
hg ci -Am r1
cd ..\Repo2
echo >file4
echo >file5
echo >file6
hg ci -Am r2
cd ..\Repo3
echo >file7
echo >file8
echo >file9
hg ci -Am r3
cd ..
@REM Create a global repo.
hg init globalRepo
cd globalRepo
@REM Pull in the first repo and move the files in it to a module1 subdirectory.
@REM Force the pull since it is an unrelated repo and update to it.
hg pull ..\Repo1 -f
hg update tip
md module1
hg rename * module1
hg ci -m "Rename Repo1 files to module1 directory."
@REM Pull in the second repo and move the files in it to a module2 subdirectory.
hg pull ..\Repo2 -f
hg update tip
md module2
hg rename * module2
hg ci -m "Rename Repo2 files to module2 directory."
@REM Merge the two unrelated branches together.
hg merge
hg ci -m "Merge the two unrelated repositories together."
@REM Pull in the third repo, rename, and merge.
hg pull ..\Repo3 -f
hg update tip
md module3
hg ren * module3
hg ci -m "Rename Repo3 files to module3 directory."
hg merge
hg ci -m Merge

The result is shown below. Note that the entire history of the original three projects will each start with a root node with no parent. Files are moved to their final directory destination in each project, then merged together.