CODEWITHSUNDEEP
character-encodingsublimetext2sublimetextsublimetext3
blue note
blue note

Reputation: 29081

Sublime Text: Not representable characters

I'm using Sublime Text for Latex, so i need to use a specific encoding. However, in some cases, when I paste text copied from a different program (word/browser in most cases), I'm getting the message:

"Not all characters are representable in XXX encoding, falling back to UTF-8"

My question is: Is there any way to see which parts of the text cannot be encoded, so I can delete them manually?

Upvotes: 12

Views: 10277

Answers (3)

Juan Javier Triff Cabanas
Juan Javier Triff Cabanas

Reputation: 564

Just adding to @Draken response: here is the RegEx with spanish characters added.

[^a-zA-Z0-9 -\.;<>/ =“”'{}\[\]\^\?_\\\|:\r\n~@àèêîôâûçäöüÄÖÜßáéíóúñÑ¿€]

In my case I hitted Ctrl+H (for replacement) and as a replacement expression used nothing. So everything got cleared super fast and I was able to save it using ISO-8859-1.

Upvotes: 1

LexeY4eg
LexeY4eg

Reputation: 61

For Linux users, it's also possible to automatically remove broken characters with command iconv:

iconv -f UTF-8 -t Windows-1251 -c < ~/temp/data.csv > ~/temp/data01.csv

-c Silently discard characters that cannot be converted instead of terminating when encountering such characters.

Upvotes: 3

Draken
Draken

Reputation: 280

I had this problem. It is caused by corrupt characters in your document. Here is how i solved it.

1) Make a search in your document for all standard characters. Make sure you enable regular expressions in your search, then paste this :

[^a-zA-Z0-9 -\.;<>/ ={}\[\]\^\?_\\\|:\r\n@]

You can add to that the normal accented characters of your language, here are the characters for French and German. Such as éà and so on :

[^a-zA-Z0-9 -\.;<>/ ='{}\[\]\^\?_\\\|:\r\n~@éàèêîôâûçäöüÄÖÜß]

2) Search for that, and Keep pressing F3 until you see mangled characters. Usually something like "è" which is a corrupt version of "à".

3) Delete those characters or replace them with what they should be.

You will be able to convert the document to another encoding when you have cleared all corrupt characters out.

Upvotes: 17

Related Questions