CODEWITHSUNDEEP
regexbashperlrenamebatch-rename
user12447773
user12447773

Reputation: 11

replace part of file name with wrong encoding

Need some guidance how to solve this one. Have 10 000s of files in multiple subfolders where the encoding got screwed up. Via ls command I see a filename named like this 'F'$'\366''ljesedel.pdf', that includes the ' at beginning and end. That's just one example where the Swedish characters åäö got wrong, in this example this should have been 'Följesedel.pdf'. If If I run #>find . Then I see a list of files like this: ./F?ljesedel.pdf

Not the same encoding. How on earth solving this one? The most obvious ways:

  myvar='$'\366''
  char="ö"
  find . -name *$myvar*   -exec rename 's/$myvar/ö'  {} \;

and other possible ways fails since find . -name cannot find it due to the ? instead of the "real" characters " '$'\366'' "

Any suggestions or guidance would be very much appreciated.

Upvotes: 1

Views: 556

Answers (2)

dash-o
dash-o

Reputation: 14432

I'm not an expert - but it might not be a problem with the file name (which seems to hold the correct Unicode file name) - but with the way ls (and many other utilities) show the name to the terminal.

I was able to show the correct name by setting the terminal character encoding to Unicode. Also I've noticed the GUI programs (file manager, etc), were able to show the correct file name.

Gnome Terminal: "Terminal .. set character encoding - Unicode UTF8

It is still a challenge with many utilities to 'select' those files (e.g., REGEXP, wildcard). In few cases, you will have to select those character using '*' pattern. If this is a major issue considering using Ascii only - may be use the 'o' instead of 'ö'. Not sure if this is acceptable.

Upvotes: 0

Grinnz
Grinnz

Reputation: 9231

The first question is what encoding your terminal expects. Make sure that is UTF-8.

Then you need to find what bytes the actual filename contains, not just what something might display it as. You can do this with a perl oneliner like follows, run in the directory containing the file:

perl -E'opendir my $dh, "."; printf "%s: %vX\n", $_, $_ for grep { m/jesedel\.pdf/ } readdir $dh'

This will output the filename interpreted as UTF-8 bytes (if you've set your terminal to that) followed by the hex bytes it actually contains.

Using that you can determine what your search pattern should be. Your replacement must be the UTF-8 encoded representation of ö, which it will be by default as part of the command arguments if your terminal is set to that.

Upvotes: 1

Related Questions