Git/Hg repository(ies) with complex projects structure

Question

at the moment we're using SVN (yeah, I know that's shame :)) and we're checking possibility to move for Git or Hg. At the moment we have one svn repo with structure like that:

Platform 1-
          . Country 1
                    . Customer A
                               . Project Z
                               . Project Y
                    . Customer B
                               . Project X
          . Country 2
                    . Customer C
                               . Project W
Platform 2 
          . Country 1

etc.

We have only 2 top level "Platforms" but number of Countries, Customers and especially projects is pretty high (I'm sure number of projects >100). This structure is quite handy for us as it is easy to find necessary project (as developer knows Platform, Country and Customer). And as usually developer can switch project few times during 1 week this organization is quite important.

Basic use case:

1. Engineer receives project description (quite often someone already worked on the project)
2. Engineer has whole SVN repo copy on his HDD. He goes to the necessary Country/Customer and does SVN Update for Customer folder (if he doesn't have project) or Project folder (if he already checked out it before).
3. User works on the project and does commits by committing whole Project folder.

Some people just browse through TortoiseSVN Repo Browser, find necessary project and check it out.

So now we need something same simple but with Git/Hg. As I understand in Git the very common pattern - 1 Repo/1 Project. So the questions are:

1. Is there any way to categorize repos in Git/Hg?
2. Is there way to browse repos throug GUI (Win) and pull necessary one?
3. Do you have any offer how to organize our repo using Git/Hg?

I created BitBucket Git repo to play with it but haven't understand now how I can organize our projects.

Just one note - branches isn't very common case for us as projects are quite small in terms of development.

Answer 1

As I understand in Git the very common pattern - 1 Repo/1 Project.

It's not just very common, it's practically mandatory when you use a distributed version control system like Mercurial or Git. The reason is that both systems don't support narrow clones yet, that is, clones where you only download the history belonging to a certain subdirectory in the bigger repository. So you need to checkout everything just to get a small part of the repository.

So create one repository for each project. You can organize the projects into groups by creating a folder structure like you have today. The folder structure won't be in any repository — it's just a way to group related repositories.

Tools such as Kallithea will also let your organize repositories in hierarchical groups. This will let your users browse the repositories in a convenient way.

So, the questions were:

1. Is there any way to categorize repos in Git/Hg?
2. Is there way to browse repos throug GUI (Win) and pull necessary one?
3. Do you have any offer how to organize our repo using Git/Hg?

To sum up the answer above: Yes, you can organize repositories. You can do it on the file system level by putting them into a directory structure and you can do it on the HTTP level by using a suitable repository manager.

How to browse the repositories to find the right one will depend on the organization. The recommended way to share repositories is HTTP and then the "Open Repository" dialog won't let you browse around in the folders — since the folders only exist in the URLs served by, say, Kallithea. Your engineers will therefore typically browse the repository structure in their browser and then copy-paste the right URL into TortoiseHg.

When they clone from the server, the engineer should then put the local clones into a hierarchy that match the one on the server. This hierarchy will be done with directories on their local disk. They can then of course browse this hierarchy with Windows Explorer like normal.

Answer 2

You want one repo per project. This keeps down the amount of data you need to keep local when working on a project lower: unlike SVN, you have the complete history of every change in every repo copy. It also keeps commits on one project from getting interspersed with commits from another project when doing git logs.

Then you add a top level repo to handle your top level structure. In that repo, you add git submodules for each of the leaf repos. People always start cloning/pulling the top level structure then only check out the projects they need with git submodule update and git pull for the projects.

The top level project will track the head hashes of the submodules which you can use or not depending on your preference.

Answer 3

Yes, it is very common and highly recommended to have a repo per project ( repos are more lightweight in Git / Hg than in SVN). This is so that people who need to checkout (clone) only one project can do so . Otherwise, everyone will have to clone the entire set of projects. However, people who want everything, will have to clone one-by-one. So if you aren't looking at having branches, you can have all the projects in one repo ( as branching cannot be at project level if you have them all together and hence per project repo will be necessary if you were branching.)

Categorize repos based on the folder so that the URL will show the categorization like git@host.com/platform1/Country1/CustomerA/project1.git etc. Note that if you have the projects in a single repo, you will not be able to get such a URL. You will only have git@host.com/repo.git or something like that.

See the tradeoffs and decide or one repo or multiple repos. You may even want to group by platform etc. rather than projects.

You can use web interface like GitWeb or cgit to browse repos.

Answer 4

The simplest option is that you simple use one git repository for the project, just like you do with subversion.

Advantage: just the same as you have. Drawback: to update any part you update all parts of the repository; there is no "only here" update like SVN.

Generally, if the platform, country, and project are entirely independent I would suggest your engineers just create that structure locally, and you name your repositories ${platform}-${country}-${project} to preserve that detail.

You can obviously script the checkout process to build up that structure, if you wish, and you might look to tools like mr or other tools designed to handle multiple repositories from a single location, if you wish.

On the other hand, if your projects are very similar you might look at branches, and perhaps a git rebase driven workflow, which will probably make it easier to merge common changes into your system.