CODEWITHSUNDEEP
c#com
Geoffrey Wass
Geoffrey Wass

Reputation: 41

How to prevent conversion of Windows-1252 argument into a Unicode string?

I've written my first COM classes. My unit tests work fine, but my first use of the COM objects has hit a snag.

The COM classes provide methods which accept a string, manipulate it and return a string. The consumer of the COM objects is a dBASE PLUS program.

When the input string contains common keyboard characters (ASCII 127 or lower), the COM methods work fine. However, if the string contains characters beyond the ASCII range, some of them get remapped from Windows-1252 to C#'s Unicode. This table shows the mapping that takes place: http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1252.TXT

For example, if the dBASE program calls the COM object with:

oMyComObject.MyMethod("It will cost€123") where the € is hex 80,

the C# method receives it as Unicode:

public string MyMethod(string source)
{
    // source is Unicode and now the Euro symbol is hex 20AC
    ...
}

I would like to avoid this remapping because I want the original hex content of the string.

I've tried adding the following to MyMethod to convert the string back to Windows-1252, but the Euro symbol gets lost because it becomes a question mark:

        byte[] UnicodeBytes = Encoding.Unicode.GetBytes(source.ToString());
        byte[] Win1252Bytes = Encoding.Convert(Encoding.Unicode, Encoding.GetEncoding(1252), UnicodeBytes);
        string Win1252 = Encoding.GetEncoding(1252).GetString(Win1252Bytes);

Is there a way to prevent this conversion of the "source" parameter to Unicode? Or, is there a way to convert it 100% from Unicode back to Windows-1252?

Upvotes: 2

Views: 1682

Answers (2)

Geoffrey Wass
Geoffrey Wass

Reputation: 41

Yes, I'm answering my own question. The answer by "Jigsore" put me on the right track, but I want to explain more clearly in case someone else makes the same mistake I made.

I eventually figured out that I had misdiagnosed the problem. dBASE was passing the string fine and C# was receiving it fine. It was how I checked the contents of the string that was in error.

This turnkey builds on Jigsore's answer:

void Main()
{
    string unicodeText = "\u20AC\u0160\u0152\u0161";

    byte[] unicodeBytes = Encoding.Unicode.GetBytes(unicodeText);
    byte[] win1252bytes = Encoding.Convert(Encoding.Unicode, Encoding.GetEncoding(1252), unicodeBytes);
    for (int i = 0; i < win1252bytes.Length; i++)
        Console.Write("0x{0:X2} ", win1252bytes[i]); // output: 0x80 0x8A 0x8C 0x9A

    // win1252String represents the string passed from dBASE to C#
    string win1252String = Encoding.GetEncoding(1252).GetString(win1252bytes);

    Console.WriteLine("\r\nWin1252 string is " + win1252String); // output: Win1252 string is €ŠŒš
    Console.WriteLine("looking at the code of the first character the wrong way: " + (int)win1252String[0]);
    // output: looking at the code of the first character the wrong way: 8364

    byte[] bytes = Encoding.GetEncoding(1252).GetBytes(win1252String[0].ToString());

    Console.WriteLine("looking at the code of the first character the right way: " + bytes[0]);
    // output: looking at the code of the first character the right way: 128

    // Warning: If your input contains character codes which are large in value than what a byte
    // can hold (ex: multi-byte Chinese characters), then you will need to look at more than just bytes[0].
}

The reason the first method was wrong is that casting (int)win1252String[0] (or the converse of casting an integer j to a character with (char)j) involves an implicit conversion with the Unicode character set C# uses.

I consider this resolved and would like to thank each person who took the time to comment or answer for their time and trouble. It is appreciated!

Upvotes: 2

Jigsore
Jigsore

Reputation: 11

Actually you're doing the Unicode to Win-1252 conversion correctly, but you're performing an extra step. The original Win1252 codes are in the Win1252Bytes array! Check the following code:

string unicodeText = "\u20AC\u0160\u0152\u0161";

byte[] unicodeBytes = Encoding.Unicode.GetBytes(unicodeText);
byte[] win1252bytes = Encoding.Convert(Encoding.Unicode, Encoding.GetEncoding(1252), unicodeBytes);
for (i = 0; i < win1252bytes.Length; i++)
    Console.Write("0x{0:X2} ", win1252bytes[i]);

The output shows the Win-1252 codes for the unicodeText string, you can check this by looking at the CP1252.TXT table.

Upvotes: 1

Related Questions