Git pull is very slow... Why?

Question

Note I have studied the Git-is-very-very-slow question, but in their case the reason was big binary files, while in my repository there is PHP/JavaScript/HTML/CSS only code (no binaries) and the biggest file in the repository is around 800 KB.

I've changed one file (a few lines), then git add . and git commit -m "msg", then git push origin master.

On some other machine, when I do git pull origin master it downloads a few MiB of data, and it takes more than two minutes to calculate the delta and apply changes. Something is terribly wrong here.

I suspect some recent operations may cause this:

Recently, I've accidentally added many vendor assets (bower_components assets) when I realized it, I've used git rm to remove them from repository (and of course, git add, git commit and git push to upstream).

That was a few days ago and the problems I have right now started happening around that time.

I have two questions:

• Why is this happening?
• How can I fix my repository?

Note: I am the only one using and pushing to this repository.

Answer 1

I tried all the ways in this thread, but without any luck. And it came to me that I should try pulling with Visual Studio Code, as it helped me in a Git-related issue before. And it finished pulling literally in one second.

I don't have profound knowledge of how Git works. So I can't really explain why. Maybe it's because Visual Studio Code is embedded with a different version of Git.

Answer 2

I had problem with IPv4/IPv6. So using the following commands, it resolved it for me.

git pull -4
git fetch -4

Or

git pull --ipv4
git fetch --ipv4

You can also set up a ~/.ssh/config file for this with following snippet:

Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_ed25519
    AddressFamily inet

Answer 3

This could be due to the Git protocol you use. I switched the GitLab project URL to https, and the slowness nightmare was gone!

You can edit your repository .git/config and change to HTTPS URL, or via command:

git remote set-url https://{server}@gitlab.com/{user}/{project}.git`

Answer 4

I tried all solutions in this thread without any luck. I tried using Git protocol 2 at the suggestion of a coworker, which ended up being very effective (went from waiting 3 minutes for pulls/pushes to start to a few seconds):

git config --global protocol.version 2

Answer 5

Just in case if someone is stumble upon this thread, before deleting your .git folder, try to restart your Wi-Fi device. That may be just your Wi-Fi connection issue.

Answer 6

I have had the same issue when I was dealing with thousands of small files.

The thing that fixed it for me was to set the postbuffer in the Git repository's configuration:

git config http.postBuffer 524288000

Instead of uploading with 18 KB/s it suddenly went the full bandwidth.

Answer 7

I had a similar experience—Git pull and push suddenly starting to run extremely slowly, taking ten minutes or more, both on my local Mac OS X and on my Linux / Apache server. I deleted the local copy of the repository on my Mac, and recloned it, and it started to run fine. I did the same thing on the server, and all is well. I suppose it was somehow corrupted?

Answer 8

The problem was in EmberJS application directory. It contained node_modules folder and bower_components directories which kept third-party libraries used by Grunt to build my JavaScript and CSS assets.

Each of these contained many files and directories... considering that the dependency tree contained hundreds of libraries of size varying from small (few files) to big fat (many files).

After removing these directories and ignoring them, the Git repository works fast again.

Answer 9

I was working on a repository that had a lot of remote branches. One thing that helped me was remove all the remote tracking branches except origin/main:

git branch -r --no-contains "$(git rev-parse --quiet origin/main)" | xargs -I{} git branch -rd "{}"
git gc --prune=now

This reduced the size of my .git directory from 16 GiB to 4.4 GiB. Instead of git pull, I now use this:

git fetch origin main
git merge origin/main

While git fetch is still slow, it has to download so much less data that it takes much less time.

Answer 10

I was using Linux Mint and GitLab and GitHub. I had no problems with pull/fetch before, but suddenly I only had problems with GitLab. After reading this thread, I understood that it might be related to SSH and IPv4/6.

When I saw on https://whatismyipaddress.com/ that the website could not find my IPv6 address, I restarted my router. Now everything is fine.

so before you start changing setting try this simple solution

Answer 11

Not only the protocol v2 will help, but the commit graph (mentioned here) will help too.

With Git 2.34 (Q4 2021), loading of ref tips to prepare for common ancestry negotiation in "git fetch-pack"^(man) has been optimized by taking advantage of the commit graph when available.

See commit 3e5e6c6 (04 Aug 2021) by Patrick Steinhardt (pks-t).
^{(Merged by Junio C Hamano -- gitster -- in commit 1b2be06, 24 Aug 2021)}

fetch-pack: speed up loading of refs via commit graph

^{Signed-off-by: Patrick Steinhardt}

When doing reference negotiation, git-fetch-pack(1) is loading all refs from disk in order to determine which commits it has in common with the remote repository.
This can be quite expensive in repositories with many references though: in a real-world repository with around 2.2 million refs, fetching a single commit by its ID takes around 44 seconds.

Dominating the loading time is decompression and parsing of the objects which are referenced by commits.
Given the fact that we only care about commits (or tags which can be peeled to one) in this context, there is thus an easy performance win by switching the parsing logic to make use of the commit graph in case we have one available.
Like this, we avoid hitting the object database to parse these commits but instead only load them from the commit-graph.
This results in a significant performance boost when executing git-fetch^(man) in said repository with 2.2 million refs:

Benchmark #1: HEAD~: git fetch $remote $commit
  Time (mean ± σ):     44.168 s ±  0.341 s    [User: 42.985 s, System: 1.106 s]
  Range (min … max):   43.565 s … 44.577 s    10 runs

Benchmark #2: HEAD: git fetch $remote $commit
  Time (mean ± σ):     19.498 s ±  0.724 s    [User: 18.751 s, System: 0.690 s]
  Range (min … max):   18.629 s … 20.454 s    10 runs

Summary
  'HEAD: git fetch $remote $commit' ran
    2.27 ± 0.09 times faster than 'HEAD~: git fetch $remote $commit'

Answer 12

I had the same issue. For me this was a IPv4/IPv6 issue. I fixed it forcing SSH to use IPv4.

Set "AddressFamily inet" in /etc/ssh/ssh_config to force IPv4 connection. Then restart ssh client sudo service ssh restart

More info here.