Reputation: 1424

Git error, need to remove large file

I am getting this error when I try to push to git and I have no idea how to fix it.

Counting objects: 1239, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (1062/1062), done.
Writing objects: 100% (1239/1239), 26.49 MiB | 679.00 KiB/s, done.
Total 1239 (delta 128), reused 0 (delta 0)
remote: warning: File log/development.log is 98.59 MB; this is larger than GitHub's recommended maximum file size of 50.00 MB
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
remote: error: Trace: efd2d13efa4a231e3216dad097ec25d6
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File log/development.log is 108.86 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File log/development.log is 108.74 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File log/development.log is 108.56 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File log/development.log is 106.58 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File log/development.log is 103.70 MB; this exceeds GitHub's file size limit of 100.00 MB
To [email protected]:myUsername/myRepo.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '[email protected]:myUsername/myRepo.git'

I'm guessing I need to remove the large file from the commit, but how do I do that?

Upvotes: 72

Answers (18)

Eric

Reputation: 1

I don't believe this idea has been suggested above but sometimes if it gets too crazy trying to resolve the large file issue or other git issues, I've just grabbed a copy of the repo (the whole local folder etc) and put it in a folder anywhere away from git. Then do a fresh clone of the repo. Next a diffing tool like winmerge to bring in any work I don't want to lose. And leave out the offending files and push back upstream. I apologize if this is not a correct answer but it has saved a lot of hair pulling when the above just doesn't seem to work.

Upvotes: 0

Ahmad Mustapha

Reputation: 1

To resolve the error of large files, you first need to remove the large file from your repo.

Then try this code:

rm -rf .git/refs/original/
rm -rf .git/logs/
git reflog expire --expire=now --all
git gc --prune=now --aggressive

This will clear the references.

Then create a new branch by trying this code:

git checkout --orphan new-main
git commit -m "Cleaned commit history"
git push origin new-main --force

This will clear the entire history of the large file, so you can then proceed to reinitiate.

git remote add origin https://github.com/USERNAME/REPOSITORY_NAME.git

I did this and it works.

Upvotes: 0

Nestroy

Reputation: 77

Alternatively, git rebase may be used to alter the commit in the earlier commit. First, you need to identify the hash of the commit where the change happened. For instance, to modify 35da8436, you may use

$ git rebase --interactive 35da8436~

The tilde (~) is strictly necessary to reapply the subsequent commits. git rebase will now open an editor of structure:

pick 35da8436 style: add new resources for roofs and clean up some old ones
pick 09f6df0d feat/WIP!: major restructure and rewrite of components

Change pick to edit in the line of the commit to be modified. Once the file is saved, the HEAD of the repository will be at the named commit. The commit may now be modified. Repeat the steps mentioned previously:

  $ git rm --cached GIANT_FILE
  # Stage our giant file for removal, but leave it on disk
  $ git commit --amend -CHEAD
  # Amend the previous commit with your change
  # Simply making a new commit won't work, as you need
  # to remove the file from the unpushed history as well

Consequently, you may return to the original HEAD using git rebase --continue and push the smaller commits using git push.

Reference and credits to https://stackoverflow.com/a/1186549

Upvotes: 0

Hashim Aziz

Reputation: 6165

If you try to run git filter-branch with unstaged changes you'll receive a fatal error that prevents you from doing so. To solve this you need to run two commands before and after it:

git stash save

git filter-branch -f --tree-filter 'rm -f /path/to/file.log' HEAD --all

git stash pop

The first will stash away your changes safely and the second will put them back onto the staging area once filter-branch has finished.

Speaking of finishing, any filter-branch command will take a relatively long time so be prepared for this - it took just under an hour for me on a repo with ~2000 commits.

By contrast, git filter-repo performed the same operation in about a minute, and is also much safer to run:

git filter-repo -f --invert-paths --path /path/to/file.log

Upvotes: 0

Cinzia Nicoletti

Reputation: 305

Update 2024:

Go to:

Github -> [project name] -> General -> Archives

And check: Include Git LFS objects in archives

Upvotes: 2

Kyle Smith

Reputation: 46

You can run this (taken from a post above):

git reset --soft HEAD~1

Or if you don't want to use internet-provided git commands (from previous traumas, lol):

Remove the .git folder from your project.
Clone your repository into a different folder (git clone ...).
Copy the .git folder from the directory you cloned, and paste that .git folder into your project.

Both of the options does almost the same thing.

Upvotes: 0

Abayomi Olowu

Reputation: 349

i was able to solve mine with this

git reset --soft HEAD~1
git pull origin master --allow-unrelated-histories
git push -u origin master or git push

Upvotes: 0

childerino

Reputation: 401

As of October 2022, git instructs the user to use 'git filter-repo' because:

WARNING: git-filter-branch has a glut of gotchas generating mangled history
         rewrites.  Hit Ctrl-C before proceeding to abort, then use an 
alternative filtering tool such as 'git filter-repo'
         (https://github.com/newren/git-filter-repo/) instead.  See the
         filter-branch manual page for more details; to squelch this warning,
         set FILTER_BRANCH_SQUELCH_WARNING=1.

If (for some reason) the documentation for git-filter-repo (https://github.com/newren/git-filter-repo/) is not exactly clear, the following will definitely get you on track (it did for me, as I'm apparently a bit obtuse):

https://superuser.com/a/1589985/1054410

The following examples from the git-filter-repo docs show how to translate between, say:

git filter-branch -f --tree-filter 'rm -f /path/to/file' HEAD --all

git filter-repo --invert-paths --path /path/to/file

https://github.com/newren/git-filter-repo/blob/main/Documentation/converting-from-filter-branch.md#cheat-sheet-conversion-of-examples-from-the-filter-branch-manpage

Finally, you might need to add '--force' to the end of the statement, because git filter-repo expects a 'fresh clone'.

Aborting: Refusing to destructively overwrite repo history since      
this does not look like a fresh clone.
  (expected freshly packed repo)
Please operate on a fresh clone instead.  If you want to proceed      
anyway, use --force.

Upvotes: 9

CZTopp

Reputation: 141

I had already deleted the file thinking that would help. Running this command on the the branch worked for me.

git filter-branch -f --index-filter 'git rm --cached --ignore-unmatch path/to/file'

Upvotes: 14

Tejas Niturkar

Reputation: 201

This command worked for me

git filter-branch -f --index-filter 'file path'

Upvotes: 0

Timbus Calin

Reputation: 15053

Adding to the previous answers:

git filter-branch -f --index-filter 'git rm --cached --ignore-unmatch path_to_the_file/your_big_file'

Additionally, when I tried to pull I got "Refusing to merge unrelated histories".

Solution was to use:

git pull origin branch_name --allow-unrelated-histories

Then solve your possible conflicts, and push this time without the big_file.

Upvotes: 5

Guy Luz

Reputation: 4039

Adding to the previous answers:

We are going to remove the large file from the commit as you said

Do the push from the terminal just to get the large file path git push --set-upstream origin Remote_branch_name, locate the large file path in the errors, something like RepositoryName/folders.../bigFileName.
Remove the file from all the branches, git filter-branch -f --index-filter 'git rm --cached --ignore-unmatch path_and_big_file' past the path that we found in section one instead of path_and_big_file.
Execute git pull origin branch_name --allow-unrelated-histories to not get unrelated histories error
Now try to push to git, it should work

Upvotes: 4

Sithija Piyuman Thewa Hettige

Reputation: 1888

git reset --soft HEAD~1

then exclude the file from the commit.

Note: Use HEAD~N to go back to N number of previous commits.

use this , go back a commit ,remove the large file from the commit if you can see it , then re commit and push.

this should solve the problem

Upvotes: 22

user30850

Reputation: 521

To improve upon one of the reply above, you need

git filter-branch -f --tree-filter 'rm -f /path/to/file' HEAD --all

in order to remove your file even from the history. You need -f to force rewriting any backup and --all to remove file from all branches. More information here: Git docs

Your large file error won't go away by removing it from the git cache because it is still lurking in the commit history.

Upvotes: 42

Ritesh Chandnani

Reputation: 475

I faced the same issue recently, you can actually just filter the branch for specific file and remove it -

git filter-branch --tree-filter 'rm -rf path/to/your/file' HEAD

Upvotes: 28

Daniel Andrei Mincă

Reputation: 5032

To remove large files, GitHub suggests:

$ git rm --cached giant_file
# Stage our giant file for removal, but leave it on disk

git commit --amend -CHEAD
# Amend the previous commit with your change
# Simply making a new commit won't work, as you need
# to remove the file from the unpushed history as well

git push
# Push our rewritten, smaller commit

Or if you want more general information on how to work with large files on GitHub.

And next time add /log in your .gitignore

Upvotes: 65