CODEWITHSUNDEEP
git
Ram
Ram

Reputation: 461

Not able to clone large repo code on git

I'm getting an error like :-

Cloning into 'large-repository'...
remote: Counting objects: 20248, done.
remote: Compressing objects: 100% (10204/10204), done.
error: RPC failed; curl 18 transfer closed with outstanding read data remaining 
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed

Upvotes: 14

Views: 25723

Answers (7)

Jason Tumusiime
Jason Tumusiime

Reputation: 317

I gave up

remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (15/15), done.
error: RPC failed; curl 92 HTTP/2 stream 5 was not closed cleanly: CANCEL (err 8)
error: 7832 bytes of body are still expected
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output

Upvotes: 0

Bonface Odhiambo
Bonface Odhiambo

Reputation: 1

I had this same problem and two approaches worked for me:

i) Using SSH rather than HTTPS to download the repository. For this, you will need to generate a GitHub ssh key to enable you to use ssh for downloading the GitHub repository as described in this link

ii) Downloading the repository using the zipped file option after clicking the 'code' drop-down icon of the repository.

Upvotes: 0

Prabhakar Kumar Ojha
Prabhakar Kumar Ojha

Reputation: 59

In most cases, it will get occur because of slow internet. Try to clone in chank.

What should you do then?
Step-1: git clone --depth=1

Step-2: git fetch --depth=x (Here x will be an integer value which refers number-of-commits, do not give a value more than 200+ in incremental order if the internet is slow)

run this above command multiple times.

finally: git fetch --unshallow

unshallow: Git provides a fetch --unshallow command which solves the problem, so we just need to run git fetch --unshallow in the repository before running r10k. However, some of our (older) GitLab installs don't make shallow clones. Instead, they make full clones with a single detached branch, so we need to fetch --all instead

Upvotes: 0

Yves Forget
Yves Forget

Reputation: 139

Got the same error message.

The real cause was a disk full, the repo being cloned is around 20 gigs.

Clone with depth=1 worked, and then fetched depth by small increments. After a while I noticed disk was almost full.

Made up free space on the disk, deleted the partial repo and the "git clone" with full depth worked. Unfortunately the error message (The remote end hung up unexpectedly) was misleading.

Upvotes: 0

lucidbrot
lucidbrot

Reputation: 6156

First, try to download a smaller amount, so that when the network fails, you don't have to start from zero:
Taken From This Answer by ingyhere

First, turn off compression: 

git config --global core.compression 0

Next, let's do a partial clone to truncate the amount of info coming down: 

git clone --depth 1 <repo_URI>

When that works, go into the new directory and retrieve the rest of the clone: 

git fetch --unshallow

or, alternately, 

git fetch --depth=2147483647

Now, do a regular pull: 

git pull --all

I think there is a glitch with msysgit in the 1.8.x versions that exacerbates these symptoms, so another option is to try with an earlier version of git (<= 1.8.3, I think).

If this does not help, because your network is still too unstable or your repo still too large, try a different network - best would be a wired.

For me, that was not an option. VonC's Answer states to do git config --global http.postBuffer 524288000. Maybe you'll need to do git config --global https.postBuffer 524288000 instead, if you're using https.

Finally, what worked for me in the end:
Give up and use a different machine
If it works on your laptop, just pull that repo onto your laptop, then run

git bundle create /my/thumb/drive/myrepo.bundle --all

And restore it on your other machine with

git clone /my/thumb/drive/myrepo.bundle

Upvotes: 6

VonC
VonC

Reputation: 1324607

That looks like a curl error, typical of a slow internet connection which closes too soon.

As seen here, try a shallow clone (or switch to ssh)

git clone https://[email protected]/weexcel1/higher-education-haryana.g‌​it --depth 1

Even then, as I documented in 2011, you might need to raise the http.postBuffer 

git config --global http.postBuffer 524288000

But the idea remains: starting with one commit depth can help.

From there, you can gradually increase the depth:

git fetch --depth=<number-of-commits>

And, after a few iteration:

git fetch --unshallow

Upvotes: 9

Ram
Ram

Reputation: 461

git config --global http.postBuffer 524288000

git clone repo_url --depth 1

I have followed above steps and finally I have successfully cloned my code.

Upvotes: 19

Related Questions