CODEWITHSUNDEEP
phputf-8html-entitieshtml-encode
Rob
Rob

Reputation: 11388

Trouble decoding some special characters ’ “ ”

I'm trying to decode some special characters in php and can't seem to find a way to do it.

 $str = 'Thi’s i"s a’n e”xa“mple';

This just returns some dots.

$str = preg_replace_callback("/(&#[0-9]+;)/", function($m) {
    return mb_convert_encoding($m[1], "UTF-8", "HTML-ENTITIES");
}, $str);

Some other tests just return the same string.

$str = html_entity_decode($str, ENT_QUOTES, 'UTF-8');
$str = htmlspecialchars_decode($str, ENT_QUOTES);

Anyway, I've been trying all sorts of combinations but really no idea how to convert this to UTF-8 characters.

What I'm expecting to see is this:

Thi’s i"s a’n e”xa“mple

And actually if I take this directly and use htmlentities to encode it I see different characters to begin with.

Thi’s i"s a’n e”xa“mple

Unfortunately I don't have control of the source and I'm stuck dealing with those characters.

Are they non standard, do I need to replace them manually with my own lookup table?

EDIT

Looking at this table here: https://brajeshwar.github.io/entities/

I see the characters I'm looking after are not listed. When I test a few characters from this table they decode just fine. I guess the list in php is incomplete by default?

Upvotes: 3

Views: 2661

Answers (1)

zerkms
zerkms

Reputation: 255155

If you check the unicode standard for the characters you're referring to: http://www.unicode.org/charts/PDF/U0080.pdf

You would see that all the codepoints you have in your string do not have representable glyphs and are control characters.

Which means that it is expected that they are rendered as empty squares (or dots, depending on how your renderer treats those).

If it works for someone somewhere - it's a non-standard behaviour, which one must not rely on, since it is, well, non-standard.

Apparently the text you have has the initial encoding of cp1250, so you either should treat it accordingly, or re-encode entities manually:

$str = 'Thi’s i"s a’n e”xa“mple';

$str = preg_replace_callback("/&#([0-9]+);/u", function($m) {
    return iconv('cp1250', 'utf-8', chr($m[1]));
}, $str);

echo $str;

Upvotes: 5

Related Questions