CODEWITHSUNDEEP
c#arraysencodingutf-8character-encoding
Guy E
Guy E

Reputation: 1927

Convert from utf-8 byte array to string and back yields different data

I'm trying to figure out a very basic issue: I have a byte array with 8 bytes, containing data. I would like to convert it to string and then, on the "other side", to encode it back to the exact byte array. For some reason - this is not working. The value I'm using is 22.22 (double) and it represented as 8 bytes with the values: 184, 30,133,235,81,56,54,64 I would expect the byteArrayList single item (for complicity) which, in my case contains 8 bytes to be identical to the result item - they are not identical - the result contsins 14 bytes !!!! This is the result: 239, 191, 189, 30, 239, 191, 189, 239, 191, 189, 81, 56, 54, 64

It appears as if both the byteArrayList single item and the result contians some escape characters (\u001b in hexadecimal) What am I missing here ?
Here is my code:

List<byte[]> byteArrayList = new List<byte[]>();
...Here I populate the byteArrayList with 1 item - 8 bytes long
UTFEncoding encoding = new UTF8Encoding();
StringBuilder SB = new StringBuilder();
foreach(byte[] byteArrayItem in byteArrayList)//currently - for simplicity contains one item
{
    SB.Append(encoding.GetString(byteArrayItem ));
}
string items = SB.ToString();
byte[] result = Encoding.UTF8.GetBytes(items); //I would expect the result to be identical to the byteArrayList  - it is not ! it contains 12 bytes !!!!

Upvotes: 0

Views: 477

Answers (1)

Rob Napier
Rob Napier

Reputation: 299265

Not every sequence of bytes is valid UTF-8, so you can't use UTF-8 this way. You need to use an encoding that can handle an arbitrary sequence of bytes, such as Base64 or Hex.

The sequence 239, 181, 189 is the UTF-8 encoding of REPLACEMENT CHARACTER. This is the character used to indicate a decoding error.

Upvotes: 3

Related Questions