CODEWITHSUNDEEP
gitmallocclone
Gouda
Gouda

Reputation: 1005

git clone, upload-pack out of memory

I have a git repo that stores only PDFs as binaries, in total there are 70 PDFs totalling 130MB, a few of them are large (about 15MB). When I try to clone the repo to my work computer I get the error:

remote: Counting objects: 93, done.
remote: fatal: Out of memory, malloc failed (tried to allocate 36864964 bytes)
error: git upload-pack: git-pack-objects died with error.
fatal: git upload-pack: aborting due to possible repository corruption on the remote side.
remote: aborting due to possible repository corruption on the remote side.
fatal: protocol error: bad pack header

Some other git answers have claimed to have fixed this by repacking the repo. When I try to git repack --max-pack-size=5M -a -d or git gc on the server repo I get the same malloc error.

The git server is on my personal webspace offered by 1and1, I have the idea that it is not allowing me to use 36864964 bytes of memory in the process.

How can I clone the server repo on to my local computer?

Here is the output of ulimit -a run on the server:

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) 131072
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 48165
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) 1800
max user processes              (-u) 90
virtual memory          (kbytes, -v) 131072
file locks                      (-x) unlimited

Thank you.

Upvotes: 2

Views: 3566

Answers (1)

Gouda
Gouda

Reputation: 1005

I tried nif's comment and downloaded the repo via sftp. I then repacked it at a very low packsize using git repack --max-pack-size=3M -a -d. I then uploaded it back to the server.

At this point I got another error: fatal: unable to create thread: Resource temporarily unavailable. At this point I am almost certain that this was a resource allocation problem resulting from my host's low-memory (and perhaps threads) configuration.

After finding this stackexchange question: git push fatal: unable to create thread: Resource temporarily unavailable and setting:

git config pack.windowMemory "15m"
git config pack.SizeLimit "3m"
git config pack.threads "1"

(numbers chosen arbitrarily low)

I can now successfully clone the server repo from my local computer! I suspect that if I had configured git with the three lines above from the beginning I would not have had this problem.

Once again thank you nif!

Upvotes: 2

Related Questions