CODEWITHSUNDEEP
gitfileobjectbinary
Jonas H.
Jonas H.

Reputation: 2491

How to remove unused objects from a git repository?

I accidentally added, committed and pushed a huge binary file with my very latest commit to a Git repository.

How can I make Git remove the object(s) that was/were created for that commit so my .git directory shrinks to a sane size again?

Edit: Thanks for your answers; I tried several solutions. None worked. For example the one from GitHub removed the files from the history, but the .git directory size hasn't decreased:

$ BADFILES=$(find test_data -type f -exec echo -n "'{}' " \;)

$ git filter-branch --index-filter "git rm -rf --cached --ignore-unmatch $BADFILES" HEAD
Rewrite 14ed3f41474f0a2f624a440e5a106c2768edb67b (66/66)
rm 'test_data/images/001.jpg'
[...snip...]
rm 'test_data/images/281.jpg'
Ref 'refs/heads/master' was rewritten

$ git log -p # looks nice

$ rm -rf .git/refs/original/
$ git reflog expire --all
$ git gc --aggressive --prune
Counting objects: 625, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (598/598), done.
Writing objects: 100% (625/625), done.
Total 625 (delta 351), reused 0 (delta 0)

$ du -hs .git
174M    .git
$ # still 175 MB :-(

Upvotes: 117

Views: 92419

Answers (9)

Sam Watkins
Sam Watkins

Reputation: 8359

I answered this elsewhere, and will copy here since I'm proud of it!

... and without further ado, may I present to you this useful script, git-gc-all, guaranteed to remove all your git garbage until they might come up with extra config variables:

git -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 \
  -c gc.rerereresolved=0 -c gc.rerereunresolved=0 \
  -c gc.pruneExpire=now gc "$@"

The --aggressive option might be helpful.

NOTE: this will remove ALL unreferenced thingies, so don't come crying to me if you decide later that you wanted to keep some of them!

You might also need to run something like these first, oh dear, git is complicated!!

git remote rm origin
rm -rf .git/refs/original/ .git/refs/remotes/ .git/*_HEAD .git/logs/
git for-each-ref --format="%(refname)" refs/original/ |
  xargs -n1 --no-run-if-empty git update-ref -d

I put all this in a script, here:

https://ucm.dev/t/bin.git/git-gc-all-ferocious

Upvotes: 163

user2465896
user2465896

Reputation: 181

In 2020 the documentation for git-filter-branch discourages its use and recommends using an alternative such as git-filter-repo. It can also be used instead of BFG.

Note that the chapter on Rewriting History in the git book hasn't been updated. Neither has GitHub's recommendation on removing sensitive data.

Upvotes: 0

Czarek Tomczak
Czarek Tomczak

Reputation: 20675

See "Removing Objects" in the Pro Git book:

http://git-scm.com/book/en/Git-Internals-Maintenance-and-Data-Recovery#Removing-Objects

Update: see also BFG repo cleaner: http://rtyley.github.io/bfg-repo-cleaner/

Upvotes: 5

Jamie
Jamie

Reputation: 251

1) Remove the file from the git repo (& not the filesystem) :

  • git rm --cached path/to/file

2) Shrink the repo using:

  • git gc,

  • or git gc --aggressive

  • or git prune

or a combination of the above as suggested in this question: Reduce git repository size

Upvotes: 25

Andrew Charneski
Andrew Charneski

Reputation: 488

The key for me turned out to be running git repack -A -d -f and then git gc to reduce the size of the single git pack I had.

Upvotes: 9

Martin
Martin

Reputation: 11336

git filter-branch --index-filter 'git rm --cached --ignore-unmatch Filename' --prune-empty -- --all

Remember to change Filename for the one you want to remove from the repository.

Upvotes: 5

u-foka
u-foka

Reputation: 657

Hy!

Git only receives objects it actually needs when cloning repositories (if I understand it correctly)

So you can amend the last commit removing the file added by mistake, then push your changes to the remote repository (with -f option to overwrite the old commit on the server too)

Then when you make a new clone of that repo, it's .git directory should be as small as before the big file(s) committed.

Optionally if you want to remove the unnecessary files from the server too, you can delete the repository on the server and push your newly cloned copy (that has the full history)

Upvotes: 7

Josh Lee
Josh Lee

Reputation: 177685

Your git reflog expire --all is incorrect. It removes reflog entries that are older than the expire time, which defaults to 90 days. Use git reflog expire --all --expire=now.

My answer to a similar question deals with the problem of really scrubbing unused objects from a repository.

Upvotes: 32

Daenyth
Daenyth

Reputation: 37461

This guide on removing sensitive data can apply, using the same method. You'll be rewriting history to remove that file from every revision it was present in. This is destructive and will cause repo conflicts with any other checkouts, so warn any collaborators first.

If you want to keep the binary available in the repo for other people, then there's no real way to do what you want. It's pretty much all or none.

Upvotes: 10

Related Questions