How can I fix a missing blob in Git?

Question

Because I've been rsyncing my Git repository between various places, I'm now stuck with a broken Git repository. When I do git log I get a correct history, but when I do git status I get

fatal: unable to read 563e4c9abcd4114e08255db989f0f53426bdeff7

So after some searching around I tried git fsck:

Checking object directories: 100% (256/256), done.
missing blob 33244941016301570dccdcdc95e543910109d0a8
dangling blob 59f44441e6437ebc4d40182eb8a10d3e07fe367b
missing blob 5dc8ab1804acb58fc658bcd6152fbb246290c8ae
dangling blob 698c775f2599fad3d09906dead4dc67743a984bd
dangling blob 922003b8433bcad6ce9778a37628d738faa26389
dangling blob c33c0528bfee55b04d99de4580da49de4413329b
dangling blob e5107c118bde0edbe5dfb994cb6a50d235c3f06b
dangling blob 437573e539572454cb868ca5a0f5074b96d777ac
missing blob 468d1856336eaa1ce8006f38ce779c0d997c8d48
dangling blob 6fc9c88708d7d5ca455e68781472bdea119997eb
dangling blob 7225d0147fa566369ba3024324b527a7adeac094
dangling blob bb8125d15579fcf37925f09cd1883b15272f9f0d
missing blob c8095f49253ac3787a6f86943160eda2c78a6a28
dangling blob e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
dangling blob 0bdaed084e15add987ef86fe84f435d085475995
dangling blob 36ee13c9b334da090ea6b194606df8a2852b3b3a
missing blob 563e4c9abcd4114e08255db989f0f53426bdeff7   <= the one which results in the fatal error.
dangling blob 84f2f2a9d1d051e6418a787ca90e75446f712866
dangling blob c636d85269838efecbb496eda5a8cfd8ec753d69
dangling blob cb7a8494bfc86e894c0c6e268308ddc1dd6d713c
dangling blob d166fff9e1c85ab9f0f4f620119181c5f76c2a53
dangling blob d3b6f194df857412481a318d4275faeb6689e4a0
missing blob db9a6744bc0df03cf685296695bea6324f23e0ac
dangling blob def6a6a18457989c7d18825c7c1bbfeefc8b261d
and about 20 more..

And from here I'm kind of lost. I read something about running git reflog expire --expire=now --all, but that doesn't do anything for me.

All the files in my repository are still present and safely backed up, so that's not a problem. I would like to get my repository history back though. What steps can I take from this point?

Answer 1

If you have a good copy of the repository somewhere else, you can often use that to recover the missing objects. You might use this approach if you wanted to recover the repository without completely re-creating it from the known good repository because you, for example, want to limit the amount of data you transfer over a network.

You should make a backup copy of the bad repo and work on that first, to validate that you can recover the repo.

Let's assume the good repo is at /repo/good and the backup you're working on is at /repos/bad-backup.

1. Use git fsck --strict --connectivity-only to get a report of the missing objects:

   cd /repos/bad
   git fsck --strict --connectivity-only | awk '$1 == "missing" { print $3 }' > /tmp/missing-objects
   

2. In the good repo, create a pack file that contains the missing objects:

   cd /repos/good
   mkdir /tmp/fix-packs
   git pack-objects /tmp/fix-packs/fix1 < /tmp/missing-objects
   

3. In the bad repo, test that you can unpack the objects:

   cd /repos/bad-backup
   git unpack-objects -n /tmp/fix-packs/fix1/*.pack
   

4. If that works, in the bad repo, unpack the objects:

   cd /repos/bad-backup
   git unpack-objects /tmp/fix-packs/fix1/*.pack
   

5. Re-run git fsck --strict --connectivity-only and see if there are any missing objects. If there are missing objects, check that they are different from the ones reported in step 1. If they are new objects, repeat the steps, using a different directory in step 2, like /tmp/fix-packs/fix2/.
   • If some of the missing objects that you recover are commits, trees, tags, &c., those could point to objects that are also missing, but you won't know that they are missing until you recover the objects that point to them.

If you are working on two different machines, you will need to transfer the missing-objects file(s) created in step 1 to the machine with the good repo and you will need to transfer the .pack files created in step 2 back to the machine with the bad repo. (Steps 1, 3, 4, and 5 should be run on the machine with the bad repo. Step 2 should be run on the machine with the good repo.)

At this point, you could have a few options.

• You could swap the /repos/bad-backup for /repos/bad (e.g., mv /repos/bad /repos/to-delete-was-bad /repos/bad-backup /repos/bad).
• You could run git unpack-objects on all the pack files you created in your various executions of step 2. (Something like find -name '*.pack' /tmp/fix-packs -exec git unpack-objects -n '{}' ';') After that, git fsck --strict --connectivity-only should be clean.

I would recommend running a git fsck --full to check for any other corruption issues at this point too.

There are probably ways to streamline this with lower level Git commands. I've not taken the time to figure out how...

Answer 2

I got a "missing blob" after trying to fix "object file is empty" error (actually I ended up with object file .git/objects/f7/880aa1d1a76bfff73f3d602e15b4bc829d6a07 removed from the file system).

In order to solve the issue I followed these steps:

1. Use the Bash script found here in order to detect a commit containing this missing blob. Put it in the repository's root directory with the find.sh name:

   #!/bin/sh
   obj_name="$1"
   shift
   git log "$@" --pretty=format:'%T %h %s' \
   | while read tree commit subject ; do
       if git ls-tree -r $tree | grep -q "$obj_name" ; then
           echo $commit "$subject"
       fi
   done
   

   And then run it, passing the SHA-1 hash of the missing blob as the argument:

   ./find.sh f7880aa1d1a76bfff73f3d602e15b4bc829d6a07
   
   629afc4 ESLint warning in layers' configuration file is fixed.
   

   629afc4 is part of the commit's SHA-1 (it was a last commit I tried to push to remote repository).

2. Find a file associated with this blob:

   git ls-tree -r 629afc4 | grep f7880aa1d1a76bfff73f3d602e15b4bc829d6a07
   
   100644 blob f7880aa1d1a76bfff73f3d602e15b4bc829d6a07    src/config/layers.js
   

   It's src/config/layers.js in my case.

3. Check whether hash of the file matches the hash in Git tree:

   git hash-object src/config/layers.js
   
   f7880aa1d1a76bfff73f3d602e15b4bc829d6a07
   

4. If so then we can write file contents to the blob:

   git hash-object -w src/config/layers.js
   

Doing these steps helped me to remove the error and fix a broken local repository. The solution is found in this article.

Answer 3

The accepted answer helped me to fix the problem.

However, I suggest a faster fix if missing blobs are files that are in current directory (that was my case).

This means that, for a reason, a file has not been correctly indexed by git, and is causing the missing blob.

To find a missing blob of 04da887681cecfd6cd59a928008b0e52ffe2a2bf, you can go to the .git directory, and launch :

find . -type f -print -exec git hash-object {} \; | grep -B1 04da887681cecfd6cd59a928008b0e52ffe2a2bf

This will go through your data to find the file that is not indexed. If it find something, you now have the file to index:

./myfile.php
04da887681cecfd6cd59a928008b0e52ffe2a2bf

Then, you can index it with: git hash-object -w ./myfile.php

If it doesn't find the file, this means that it was perhaps a previous version of the file, or a file that has been lost.

Answer 4

The accepted answer or any of the variant did not work for me as git fsck did not show the missing blob neither did

$ git ls-tree -r HEAD | grep <missing blob hash id>

return anything.

What worked for me was a little hack I employed. I am sharing it in case someone comes across it.

I cloned the repo in a new location and checked it out to the branch I was working on. I shelved the changes in the current corrupted repo (as I had a few changes I could not afford to lose) and then copied over the .git folder from the newly cloned repo to the old repo. Then I ran git pull which then worked.

Answer 5

I know this is a fairly old question but I had the same problem just now and none of the other SO answers worked for me, neither did the answer provided here.

Reading your question, I got an idea from your first sentence:

Because I've been rsyncing my git repo between various places I'm now stuck with a broken git repo.

I am doing the same thing using an USB stick for synchronizing. This way I can just git pull <path/to/usb/repo> master in my working directory on any of my devices to pull from USB stick, and to "push" to USB stick I change directory to USB stick repository and use git pull <path/to/working/directory> master. It worked fine several times until just today. While doing git pull <path/to/usb/repo> master in one of my working directories, I got

errror: git upload-pack: git-pack-objects died with error.
fatal: git upload-pack: aborting due emote: fatal: unable to read 8f6d079cdb5243f5a2d1000e31967f4c361b7966
remote: aborting due to possible repository corruption on the remote side.

When doing git fsck in my USB repo, I got missing blob ...... (sorry I didnt copy that but it was just 2 missing blobs). All in all, my problem was very similar to yours. And reading yours I got the following idea:

TL;DR: In the end I did the Windows scan and fix thing as shown here, which actually pops up every single time I plug in any of my USB sticks and never did anything I found useful - until now. It actually fixed the whole problem.