CODEWITHSUNDEEP
gitgit-lfsbfg-repo-cleanersvn2git
Peter Reiser
Peter Reiser

Reputation: 11

GIT LFS migration using BFG increase total repository size by 4x

We are migrating a 5GB repository from SVN to GIT

SVN to GIT migration

svn2git http://xxx/svn/yy --trunk trunk --nobranches --notags --username user --metadata --authors /xx/yyy/authors.txt

LFS migration

java -jar ../bfg-1.14.0.jar --convert-to-git-lfs "*.{png,zip,jar,xls,xlsx,ppt,pptx,fmb,pll,ttf,afm,pfa,ttf,rdf,jpg,gif,tif,doc,docx,pdf,}" --no-blob-protection

git reflog expire --expire=now --all && git gc --prune=now

We also tried

git reflog expire --expire=now --all && git gc --prune=now --aggressive but this was taking a long time and crashed after 99%

After BFG migration the GIT repository has a total of 25 GByte and the .git/lfs store has 24 Gbyte

Why ist the LFS store getting so big and how could we reduce the size ?

Upvotes: 0

Views: 484

Answers (1)

bk2204
bk2204

Reputation: 76874

There are a couple differences between how Git stores data and how Git LFS stores data.

Git usually stores the entire contents of the repository, and in most cases, it's stored compressed and deltified (that is, with similar files as differences against other similar files). Git LFS stores files uncompressed and undeltified, but in normal usage, it only checks out files from the server that are needed for the current working tree

In this case, because you're doing a conversion, you're going to have all of the Git LFS objects on disk, and it will be large. However, after you push them to the server, in normal use, a user will likely download only a fraction of those files when they check out the repository, so the size will likely be much smaller. If you're using Git LFS 2.10.0 or newer, you can do a git clone from the large repository using a file:/// URL and will likely see that the checkout is much smaller.

Upvotes: 0

Related Questions