CODEWITHSUNDEEP
github
Dave Sims
Dave Sims

Reputation: 775

Issues with pushing large files through Git

Currently when I try to push to a Git repo, I am getting the following error.

remote: error: GH001: Large files detected.
remote: error: Trace: 7bbfe5c1099cfe679aa3cd1eee13e10a
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File e3384023be667de7529538b11c12ec68.201307290946.sql.gz is 125.37 MB; this exceeds GitHub's file size limit of 100 MB

I have gone through and made sure that this file does not exist in the directory, and have done git add -u. We have tried to prune the branch but this doesn't work as it can't find the file to delete.

Upvotes: 47

Views: 59389

Answers (8)

Ilias Iusupov
Ilias Iusupov

Reputation: 1

I addressed the issue by creating a new branch and adding a .gitignore file to exclude Terraform files. After committing these changes, I uploaded the repository again. The problem arose from Git attempting to upload a massive file when running the Terraform project. To resolve this, I created a new repository, added a .gitignore file to ignore the folder containing Terraform files. This ensured that only the essential project files, relevant to my work, were uploaded to Git, avoiding unnecessary large file uploads.

Do you get what I mean?

Upvotes: 0

Pran
Pran

Reputation: 181

  1. Download file from:https://git-lfs.github.com/
  2. Commands:

a. git lfs install

b. git lfs track "*.psd"

c. git add .gitattributes

  1. Commit and push.

Upvotes: 1

VonC
VonC

Reputation: 1329032

It is possible that you are pushing several commits, one of them including a large file, and another more recent one removing that file.

2020-2022:

Use git filter-repo (python-based, to be installed first)

And use some content-based filtering:

If you want to filter out all files bigger than a certain size, you can use --strip-blobs-bigger-than with some size (K, M, and G suffixes are recognized), e.g.:

git filter-repo --strip-blobs-bigger-than 10M

Original answer, using obsolete tools like git filter-branch or BFG:

In any case, you can try, as explained in "Fixing the “this exceeds GitHub’s file size limit of 100 MB” error", a filter-branch (if you know the name/path of the large file that you can't see)

git filter-branch --index-filter 'git rm --cached --ignore-unmatch e3384023be667de7529538b11c12ec68.201307290946.sql.gz' <sha1>..HEAD

Or, if you don't know but want to get rid of any large file (say > 90MB), you can use the BFG repo cleaner

bfg --strip-blobs-bigger-than 90M  my-repo.git

That will track for you that elusive large file in your repo history and remove it.
Note that you will have to do a git push --force after that, because the history of the more recent commits will have been modified.
If others already cloned your repo before, a bit of communication is in order to warn them.

Upvotes: 49

user9869932
user9869932

Reputation: 7377

In my case I fixed it with this link:

GitHub Help | Working with large files

Notice where it says giant_file

$ git filter-branch --force --index-filter \
  'git rm --cached --ignore-unmatch giant_file' \
  --prune-empty --tag-name-filter cat -- --all

$ git commit --amend -CHEAD

$ git push

Upvotes: 31

Vishrant
Vishrant

Reputation: 16698

Following are the steps with which you can push objects larger than 100MB:

  1. Install git-lfs instructions can be found at https://git-lfs.github.com/

  2. Let say you have *.jar files that are larger than 100 MB, then you run git lfs track "*.jar. You should see something like

    Tracking "*.jar"

  3. git add .

  4. git push -u origin master

You should see something like:

Uploading LFS objects:  89% (398/447), 864 MB | 1.8 MB/s

Upvotes: 4

Abdul Basit
Abdul Basit

Reputation: 1057

Remove file using filter-branch 

git filter-branch --tree-filter 'rm -rf your/file/path/task.txt' HEAD

make sure you must add rm -rf

after that push the code 

git push origin master -f

Upvotes: 12

SuperNova
SuperNova

Reputation: 27486

First the unstaged commits need to be resolved. Use git status

Commit these files

git status -s | grep D

Commit these files

git status -s | grep M

and then run this

git filter-branch -f --index-filter 'git rm --cached --ignore-unmatch path-to-large-file' --prune-empty --tag-name-filter cat -- --all

Once the above command is executed, the history of commits will be iterated to find the large file and will be removed.

Upvotes: 2

Kemin Zhou
Kemin Zhou

Reputation: 6891

In case you have several large files you need to run another command:

git filter-branch --index-filter 'git rm --cached --ignore-unmatch largfile1'
git filter-branch --index-filter 'git rm --cached --ignore-unmatch largefile2'
Cannot create a new backup.
A previous backup already exists in refs/original/
Force overwriting the backup with -f

You need to run

git update-ref -d refs/original/refs/heads/master

Then you can run git filter-branch command

Hope this helps you

Upvotes: 6

Related Questions