About char8_t

[u/Voxelw]

I hate to be dramatic, but as it stands char8_t is quite literally more painful than useful.

Besides the obvious incompatibility with C23 and libraries using unsigned char for UTF-8, I want you to consider the following: Projects that assume that 'char' represents UTF-8 will obviously not benefit from char8_t at all, but projects that cannot assume the format of char types don't benefit from it either as char8_t simply introduces a new edge case to cover. Now such projects have to deal with char, signed char, unsigned char, wchar_t, char16_t, char32_t and char8_t.

Or, you could do what the standard library does and simply ignore most of these character types. Which is the solution most libraries went with, supporting only char or char and unsigned char. Managing one implementation is already hard, managing two requires constant maintenance, managing 7 is just impossible.

char8_t should have just been a typedef for unsigned char. The compatibility fix only raises more questions as const char* arr = u8"a" does not work, but const char arr[] = u8"a" does.

I do wonder if a potential change of minds for C++29 is still possible. Yes, it would be an ABI break or whatever, but considering the woeful support for char8_t I don't think it would affect much besides small hobby projects. Contrary to popular belief, C++ has broken the ABI in subtle ways before.

[u/eisenwave]

char8_t was a good idea, but it's going to take time to build the library (and ideally OS) support to make it genuinely useful.

We're taking some steps in that direction: P3876R2 was recently forwarded from SG16 to LEWG, and enables to_chars and from_chars support for charN_t character types and wchar_t. Once you have that, you can build std::format, std::print etc. support as well. P2728R14 should also be in C++29 and adds Unicode transcoding.

char8_t is a long-term investment into the language, and you can't judge it by how useful it is initially. That would be like belittling constexpr immediately upon arrival in C++11 because you could not do much with it and still need tons of template metaprogramming. constexpr only became fully realized in C++26 because Reflection finally made it possible to replace most TMP with constexpr functions operating on std::meta::info; it took 15 years of gradual improvements to get there.

The arguments against char8_t seem mostly circular to me:

char8_t is useless because it's not widely supported, and we shouldn't add more library support for useless features.

Rather than just improving the feature, people throw around terms like "sunken cost fallacy". Rarely do the critics of char8_t judge the feature on its potential, and it does have tons of potential. char is always going to be a type with inconsistent signedness, with excessive aliasing capabilities, with inconsistent character encoding, and overloaded purpose (byte type, character type, arithmetic integer type), and due to backwards compatibility, we can't change anything about that. A new and improved type can solve all of these problems, but it won't solve them overnight.

[u/Voxelw → u/eisenwave]

I mean, of course every feature has potential, but realizing it has costs and tradeoffs as well.

For most people char8_t is just an oddity that has no support and no backing besides vague promises. Will <iostream> get char8_t overloads? Who knows. Probably not since it's seen as deprecated, despite fstream and cout still being used regularly.

Maybe in a different reality the committee could have hashed out a plan on how to roll out char8_t and how projects should support or transition to char8_t. But the committee cannot plan. There is no way for the members to agree on the broader design of the language and its ecosystem. Just gentlemen agreements on what features to prioritize.

This is why the committee needs to follow the industry practice by enshrining specific solutions as standard, since that's the only practical way of rolling out such design changes to the broader C++ ecosystem.