CODEWITHSUNDEEP
c++unicodefmt
Alix Blaine
Alix Blaine

Reputation: 821

Display Unicode character in C++ {fmt} library

I want to display the infinity symbol ∞ which has Unicode U+221E. I am currently using the fmt library, it is supposed to have a lot of support and be cross-platform.

fmt::print("", fmt::styled("∞ >", fmt::emphasis::bold | fg(fmt::color::aquamarine)));

I get the following output:

? >

I also tried setting: setlocale(LC_ALL, "en_US.UTF-8"); doesn't help. I am on Windows 11 x64.

WARNING:

warning C4566: character represented by universal-character-name '\u221E' cannot be represented in the current code page (1252)

MS Visual Studio 2022 IDE.

Should I change the Character Set, in project properties? Currently set to: Use Unicode Character Set, second option is: Use Multi-Byte Character Set.

Upvotes: 5

Views: 2360

Answers (4)

vitaut
vitaut

Reputation: 55524

For this to work with MSVC, you need to compile with the /utf-8 flag. Among other things, this sets the string literal encoding to UTF-8. It is detected by {fmt}, which uses Unicode APIs to write to a console. Changing the locale won't help in general, because the problems are elsewhere:

  1. Your string literal encoding (sometimes known as the execution encoding) doesn't match your source encoding - this is what causing the warning.
  2. Console uses a separate codepage (encoding).

Using wide strings won't help either, for similar reasons.

Upvotes: 2

Remy Lebeau
Remy Lebeau

Reputation: 595305

The problem is during compilation, not runtime. You are specifying the ∞ character as-is in a string literal, but the compiler is parsing your source code using codepage 1252, which does not support Unicode U+221E (and that character can't be stored in a single char anyway). So, you are losing ∞ during compiling, it is being replaced with ?, before the fmt library ever sees it.

setlocale() has no effect on this issue since it is only processed at runtime.

And the project's Character Set option has no effect on this issue either, because it only affects the compilation of TCHAR-based APIs, nothing else.

So, you have 2 choices:

  • make sure your source code is saved in UTF-8 format, and compile it with the /utf8 compiler flag specified.

  • use wide strings instead, eg:

fmt::print(L"", fmt::styled(L"∞ >", fmt::emphasis::bold | fg(fmt::color::aquamarine)));

Upvotes: 0

Phoned_Leek25
Phoned_Leek25

Reputation: 72

You're error is C4566. This is arising because Unicode formatting, especially for non standard ASCII characters goes crazy with C++.

This: char inf = '∞' breaks because not enough bits are allocated. same thing happens with string inf = '∞', string inf = L'∞', and so on..

The most convenient thing to do is to use wide characters (wchar_t) with an array to be able to breathe. There is probably another way of doing this, but this is the method I have gotten to work with the inifity symbol/Unicode symbols.

wchar_t inf[] = L"\u221e"; // ∞ in unicode is U+221E

Narrow strings (one-byte characters) are converted to multi-byte characters whereas wide strings (two-byte characters) are not.T his works because the L converts the string to a wide literal, which in layman's terms makes space for more bits so they don't go weird..

Upvotes: -1

Zenul_Abidin
Zenul_Abidin

Reputation: 809

First of all, there are different types of Unicode formats. It is likely that you are trying to print the UTF8 version of the infinity sign, but Windows uses UTF16 by default so you get crazy text instead. It actually has nothing to do with fmt.

For starters, on C++11 and later, you can change the character set used by cout, cerr, etc. to UTF8 using this code:

#include <locale>

template <typename StreamT>
void set_utf8_locale(StreamT& stream) {
    std::locale loc(std::locale(), new std::codecvt_utf8<typename StreamT::char_type>());
    stream.imbue(loc);
}

set_utf8_locale(std::cout);

Then you need to make sure your compiler is storing character strings as UTF8. You can do that by prefixing u8 before the string, such as u8"abc".

In C++20 and later, this yields a different type from const char*, so you need to use reinterpret_cast<const char*>(u8"abc") to get the correct type.

Linux and MacOS use UTF8 for output so you don't need to worry about this for those platforms.

Upvotes: -1

Related Questions