CODEWITHSUNDEEP
phpregexunicodeutf-8
Andrey
Andrey

Reputation: 60065

\w in PHP preg_replace covers only second byte of UTF-8 chars

we have this code: 

$value = preg_replace("/[^\w]/", '', $value);

where $value is in utf-8. After this transformation first byte of multibyte characters is stripped. How to make \w cover UTF-8 chars completely?

Sorry, i am not very well in PHP

Upvotes: 2

Views: 5930

Answers (5)

FYA
FYA

Reputation: 402

Append u to regex, to turn on the multibyte unicode mode of PCRE:

$value = preg_replace("/[^\w]/u", '', $value);

Corollary

In unicode mode, PCRE expects everything is multibyte and if it is not then there will be problems meeting deadlines. Therefore, to convert anything to UTF-8 (and drop any unconvertible junk), we first use:

$value = iconv( 'ISO-8859-1', 'UTF-8//IGNORE//TRANSLIT', $i );

to clean and prep the input.

Because everything can be encoded into ISO-8859-1 (even if some obscure characters appear incorrectly), and since most web browsers run natively in 8859 (unless told to use UTF-8), we've found this function as a general, safe, effective method to 'take anything, drop any junk, and convert into UTF-8'.

mb_ereg_* is deprecated as of 5.3.0 -- so using those functions is not the right way to go.

Upvotes: 2

soulmerge
soulmerge

Reputation: 75714

There is this nasty u modifier to pcre patterns in PHP. It states that the regex is encoded in UTF8, but I found that it treats the input as UTF8, too.

Upvotes: 4

Gordon
Gordon

Reputation: 316989

You could try with the /u modifier:

This modifier turns on additional functionality of PCRE that is incompatible with Perl. Pattern strings are treated as UTF-8. This modifier is available from PHP 4.1.0 or greater on Unix and from PHP 4.2.3 on win32. UTF-8 validity of the pattern is checked since PHP 4.3.5.

If that won't do, try

instead.

Upvotes: 6

user35288
user35288

Reputation:

try this function instead...http://php.net/manual/en/function.mb-ereg-replace.php

Upvotes: 1

codaddict
codaddict

Reputation: 455112

Use [^\w]+ instead of [^\w]

You can also use \W in place of [^\w]

Upvotes: 0

Related Questions