CODEWITHSUNDEEP
gitgithubrepositorygitlabgit-lfs
Kaustav
Kaustav

Reputation: 751

Can't migrate large repo from gitlab to github

I am trying to migrate a private repo from Gitlab to Github. It contains some large files which are tracked using git LFS. Even after that, I am not able to push the repo to GitHub. The size of the repo is huge as it's the full codebase for an operating system and size of the repo is around 75GB.

When I am trying to push it to GitHub, I'm getting the following logs.

Uploading LFS objects: 100% (8600/8600), 5.9 GB | 0 B/s, done
Enumerating objects: 472049, done.
Counting objects: 100% (472049/472049), done.
Compressing objects: 100% (302043/302043), done.
kex protocol error: type 7 seq 16549), 1020.66 MiB | 4.88 MiB/s
kex protocol error: type 7 seq 32979), 1.99 GiB | 4.83 MiB/s
remote: fatal: pack exceeds maximum allowed size
fatal: the remote end hung up unexpectedly
fatal: the remote end hung up unexpectedly

Please let me know if there is any way to bypass the size limit.

Upvotes: 3

Views: 3192

Answers (1)

bk2204
bk2204

Reputation: 76694

GitHub does not allow a single push to exceed 2 GB to prevent certain types of DoS attacks. You'll need to push this repository incrementally unless you're using the GitHub import utility.

You can do that by running something like this:

git rev-list --reverse --all | ruby -ne 'x ||=0; x += 1; print $_ if x % 30000 == 0;' | xargs -I{} echo git push github +{}:refs/heads/master
git push github +master
git push --mirror github

This pushes 30000 commits at a time. Once those are all pushed, it pushes the master branch one final time to contain the correct data, and then mirrors the rest of the repository.

Upvotes: 4

Related Questions