CODEWITHSUNDEEP
bashgitmacos
RonPringadi
RonPringadi

Reputation: 1494

Git checkout failed with "Illegal byte sequence"

I'm trying to rollback the source code to an older revision on git.

$ git checkout -f c258d300c8e85d68eeb15763eb14a059a2984f6d
fatal: cannot create directory at 'client-common/src/test/java/com/adi?caas': Illegal byte sequence

That adi?caas above is the cause of issue.

git diff --name-only c258d300c8e85d68eeb15763eb14a059a2984f6d~ c258d300c8e85d68eeb15763eb14a059a2984f6d
client-common/src/test/java/com/adi<F0>caas/client/common/utils/MockConfigurationFileBuilder.java

Which also could be a non standard character adi<F0>cass. How do I overcome this issue? Is there a way to automatically map this character to /. Because that is what it should be, I do have client-common/src/test/java/com/adi/caas/.

I'm using mac OSX Mojave 10.14.6 and git version 2.16.2.

Upvotes: 2

Views: 3354

Answers (3)

Jacob
Jacob

Reputation: 595

As other answers have said, it seems that the OS where the file was created allows file names that your OS doesn't. When you checkout, git tries to create a file with an illegal name, and it crashes and bails.

Here's the workaround I used: I didn't need anything in the folder containing the file with the illegal byte sequence (in your case, client-common/src/test/java/com). So I enabled git sparse checkout, which allows you to configure a list of files or directories to skip during checkout. (Note: technically, you specify which files/folders to opt into, but we'll configure it to checkout everything EXCEPT the problematic directory)

Do the following in a shell in the root directory of your repo:

# first, turn on sparse checkout in the git config
git config core.sparseCheckout true

# check if the .git/info directory exists
ls .git/info

# if you got a "No such file or directory," create it. Otherwise, skip this step
mkdir .git/info

# save our avoid-directory in a variable to make the next line more readable
# (replace the stuff in quotes with your avoid-directory)
avoidDir="client-common/src/test/java/com"

# create the list of directories to ignore and add avoidDir
# (depending on your system, you may have to add a \ before the ! )
echo -e "/*\n!$avoidDir" >> .git/info/sparse-checkout

# now you can got checkout/reset/whatever and git won't attempt to create the file with the illegal name
git checkout -f c258d300c8e85d68eeb15763eb14a059a2984f6d

Upvotes: 0

bk2204
bk2204

Reputation: 76429

On macOS, all file names are encoded in a particular normalization of UTF-8. It isn't possible to create a file with a name that isn't valid UTF-8, such as a filename with only ASCII characters and an 0xF0 byte.

However, on most Unix systems, file names are encoded as an arbitrary series of bytes, and so Git lets you name your files with any sequence of non-NUL bytes. Someone in the past put a non-UTF-8 file name into your repository and now that revision can't be checked out.

You won't be able to check out that revision on macOS at all. Git will happily check out files that overlap due to case insensitivity, but it won't allow a checkout if directories can't be created. You can try using a Linux VM or Docker container if you want to check it out, or you can use git archive to produce a tarball and use an archive tool to extract the pieces that you want into an appropriate directory. You could also create a copy of that revision with the fixed path (using git mv) on a Linux system and then check it out on your Mac.

Upvotes: 6

1218985
1218985

Reputation: 8012

It looks like violation of (UTF-8) encoding rules caused the original problem. Can you please try setting LC_CTYPE to C as shown below (This causes each byte in strings to be its own character without applying any encoding rules):

LC_CTYPE=C LANG=C git checkout -f c258d300c8e85d68eeb15763eb14a059a2984f6d

Then correct the package name as per Java Naming Conventions and commit it to avoid similar issues in future.

Upvotes: 1

Related Questions