CODEWITHSUNDEEP
javacharacter-setencodingutf-8unicode
ankush chauhan
ankush chauhan

Reputation: 1

How to Encode ascent character into UTF-8 without losing its representation?

Method:

public static void test() throws IOException {
        String input = "ABCÉ Ôpqr"; // charsetName = ISO-8859-1
        
        String utf8String = new String(StandardCharsets.UTF_8.encode(input).array());
    }

Required Output:

Output : "ABCÉ Ôpqr" Encoding : UTF-8

I wanted to convert String "ABCÉ Ôpqr" into its UTF-8 encoding without loosing its representation.

Current Output:

Output : "ABC� �pqr"

Upvotes: 0

Views: 56

Answers (1)

Remy Lebeau
Remy Lebeau

Reputation: 595295

Java's String is UTF-16 encoded. You are using StandardCharsets.UTF_8 to convert a UTF-16 String into a UTF-8 byte[] array. That part is fine. If you print out the byte values, you will see that you end up with 41 42 43 c3 89 20 c3 94 70 71 72 as expected for ABCÉ Ôpqr (É is c3 89 and Ô is c3 94).

Then, you are constructing a new String from that byte[] array, but you are not specifying the encoding of the array. This is where you are going wrong. Java needs to convert the bytes back to UTF-16, so it will use its default charset, which is clearly not UTF-8 in your case, so the byte[] array will be misinterpreted. This is why you are getting the wrong output.

For example, in ISO-8859-1, byte c3 is character à but bytes 89 and 94 have no characters defined. Whereas in Windows-1252, byte c3 is character Ã, byte 89 is character ‰, and byte 94 is character ”.

If you want the byte[] array to be interpreted as UTF-8, then you have to specify that to the String constructor, eg:

public static void test() throws IOException {
    String input = "ABCÉ Ôpqr";

    Charset utf8 = StandardCharsets.UTF_8; 
    String decoded = new String(utf8.encode(input).array(), utf8);

    // do something with decoded string...
}

Otherwise, you can use Charset.decode() instead, to match your Charset.encode():

public static void test() throws IOException {
    String input = "ABCÉ Ôpqr";

    Charset utf8 = StandardCharsets.UTF_8;
    ByteBuffer buf = utf8.encode(input);
    String decoded = utf8.decode(buf).toString();

    // do something with decoded string...
}

Upvotes: 0

Related Questions