CODEWITHSUNDEEP
git
dromodel
dromodel

Reputation: 10193

Git repository setup with binaries

My company is considering implementing Git but I have a question about what the best way would be to set it up. We have 3 sites and are planning on using Gerrit2 to create mirrors. Our repository is about 2GB and we would like to start adding binaries to it. I'm concerned about the space usage though. I don't mind if all versions of the binaries are stored in a handful of locations but I want to make sure that they don't bog down clone operations.

I understand that Git uses hard links but I think that will only work if we place a copy of the repository on every mount. Are there better options and if so what are the tradeoffs? Options that I'm looking at are "--shared" and "--reference".

Upvotes: 5

Views: 3141

Answers (4)

VonC
VonC

Reputation: 1323203

Another alternative to git media mentioned by Marcelo is git annex:

See what git-annex is not:

git-annex is not git-media, although they both approach the same problem from a similar direction. I only learned of git-media after writing git-annex, but I probably would have still written git-annex instead of using it.

Currently, git-media has the advantage of using git smudge filters rather than git-annex's pile of symlinks, and it may be a tighter fit for certain situations.
It lacks git-annex's support for widely distributed storage, using only a single backend data store.
It also does not support partial checkouts of file contents, like git-annex does.

Note: abdelsaid adds in the comments:

You can use git-annex with bup (bup allows you to have versions), see git-annex/ special remotes/ bup (and Using bup)

I have presented bup in more details in "git with large files"

Upvotes: 6

Alex Dupuy
Alex Dupuy

Reputation: 6123

Yet another possibility is git-fat (or Cyan's fork of git-fat with some enhancements); it is even lighter weight (depends only on Python and rsync) than git-media (Ruby), and definitely lighter than git-annex (Haskell). The cost of that is less configurability, but this may be the best solution in some cases.

Upvotes: 2

Adam Dymitruk
Adam Dymitruk

Reputation: 129526

To just use native git, use a separate repo to house the binaries via git submodules. This has worked for me on IVR systems which had a ton of gigantic .wav files. If you need further clarification, feel free to contact me.

Here's a good write up on them:

http://progit.org/book/ch6-6.html

hope this helps

Upvotes: 3

Marcelo Cantos
Marcelo Cantos

Reputation: 185852

Use git-media for large binaries. It stores references to binaries as SHA1 sums and hosts the binaries themselves in a place (and protocol) of your choosing. When you do a clone, it only fetches the binaries necessary to checkout the working copy.

Upvotes: 3

Related Questions