CODEWITHSUNDEEP
c++unicodetraversalwchar-twstring
yvvt0379
yvvt0379

Reputation: 63

How to traverse a wstring properly?

I was trying to traverse a std::wstring, here's my code:

#include <iostream>

int main() {
    std::wstring ws;
    std::getline(std::wcin, ws);
    for (auto wc : ws) {
        std::wcout << wc << std::endl;
    }
}

When I tried to run this program, typed “你好” into the console, the program just printed 4 blank lines.

What I expect the program to output:

你
好

I have searched this site and came back with no solution.

What should I do to produce the result I expect?

Upvotes: 0

Views: 111

Answers (1)

gerum
gerum

Reputation: 1144

First: This is an encoding problem, so it has not much connection to wstring, a string would probably have the same problem. And the size of wchar and encoding are system dependent, so your code would probably work under linux.

The explanation for your result is that under windows a wstring has 2 bytes per character and it uses UTF-16 encoding, but UTF-16 is a variable-length encoding and I am pretty sure that your (Chinese?) symbols can not be represented in 2 bytes but they need more space.

So for your exact example you could use some function or wrapper class that gives you full code points instead of code units, but I personally do not know any library that do so, because I follow my own advice:

But: I recommend to read http://utf8everywhere.org/ , especially the part about code point, code unit, abstract character and so on, and then stick to UTF-8 and the opaque data argument.

Upvotes: -1

Related Questions