CODEWITHSUNDEEP
javautf-8character-encodingansi
TheGuyNextDoor
TheGuyNextDoor

Reputation: 7937

Convert ANSI characters to UTF-8 in Java

Is there a way to convert an ANSI string to UTF using Java.

I have a custom serializer that uses readUTF & writeUTF methods of the DataInputStream class to deserialize and serialze string. If i receive a string encoded in ANSI and is too long, ~100000 chars long i get the error;

Caused by: java.io.UTFDataFormatException: encoded string too long: 106958 bytes

However in my Junit tests i'm able create a string with 120000 'a's and it works perfectly

I have checked the following posts but still having errors;

Upvotes: 4

Views: 42939

Answers (4)

István
István

Reputation: 508

ZZ Coder already answered the question, but I have written a more detailed explanation and suggesting a workaround on this blog. Basically, the problem is in DataOutputStream, because it restricts the writeable String to 64KB. There are other possible workarounds to bystep the issue, some might work without breaking the actual binary data format one is using...

Upvotes: 1

ZZ Coder
ZZ Coder

Reputation: 75456

This error is not caused by character encoding. It means the length of the UTF data is wrong.

EDIT: Just realized this is a writing error, not reading error.

The UTF length is only 2 bytes so it can only hold 64K UTF-8 bytes. You are trying to writing 100K, it's not going to work.

This limit is hardcoded and no way to get around this,

if (utflen > 65535)
    throw new UTFDataFormatException(
            "encoded string too long: " + utflen + " bytes");

Upvotes: 6

iammichael
iammichael

Reputation: 9767

byte[] asciiBytes = ...;
String unicode = new String(asciiBytes, "US-ASCII");
byte[] utfBytes = unicode.getBytes("UTF-8");

Upvotes: 3

Aaron Digulla
Aaron Digulla

Reputation: 328594

Which ANSI codepage? There are lots of different character encodings which all refer to "ANSI". The DOS codepage is 437 (without the drawing symbols). If you use codepage 850, this will work:

String unicode = new String(bytes, "IBM850");

(where bytes is an array with the ANSI characters). After that, you can convert this string into a byte array with any encoding using unicode.getBytes(encoding).

Windows often uses the codepage 1252 (use "windows-1252" for that).

Upvotes: 2

Related Questions