MySQL charset needed

Question

I'm developing an application for a native language learning. I need to store some characters as 'ẽũ'. My database is set to utf-8 charset with a default collation, also the table affected by this characters.

The problem is when I try to add a row using a regular SQL insert:

INSERT INTO text(spanish,guarani) VALUES('text','ẽũ');

This throws a warning:

Warning Code : 1366 Incorrect string value: '\xE1\xBA\xBD\xC5\xA9' for column 'guarani' at row 1

And the result is "??" where there are those characters.

Question: These characters are not covered by the UTF-8 charset? Which one I need?

Note: Same problem with latin-1

Thanks.

Answer 1

As by other answer, you really should read and understand the basics of Unicode. It's not difficult, (in one day you can grasp it), it's required knowledge for almost every programmer (and certainly for you), it's non ephemeral knowledge and will be your life simpler and happier.

These characters are not covered by the UTF-8 charset?

UTF-8 is a Unicode charset, Unicode covers (practically) every character. MYSQL's 'utf8' encoding, on the other hand, is not true UTF-8, it leaves some characters out (thouse outside the BMP). But that is not your problem here.

http://www.fileformat.info/info/unicode/char/1ebd/index.htm http://www.fileformat.info/info/unicode/char/169/index.htm

You see there that your two characters are valid Unicode, are inside the BMP (hence Mysql crippled 'utf8' should support them), and yu can even see it's UTF-8 encoding. And, as you see, \xE1\xBA\xBD\xC5\xA9 seems just right. So the problem seems to be elsewhere. Are you sure you DB is utf8?

Answer 2

QUICK!!! Read http://www.joelonsoftware.com/articles/Unicode.html

It is required reading.

Once you have read that, you should ask yourself:

1. What encoding is the connection using.
2. What locale is collation using. (If applicable).
3. What encoding is the SQL statement in?
4. What encoding are the string literals in?
5. What encoding is the html form presented in?