C++ FString and Asian Encoding

Development Pipeline & Plugins
, , , ,
1

Hello, I am in despair.
I am new to C++ code, and I’d like to know how to stock Japanese characters (like “おはよう”) in a FString.
I’ve already searched, and found that FString can stock special characters, but due to the char array, it doesn’t work.
I saw that we had to convert things, like FText to FString, but I don’t succeed to use FText (and the documentation says that it doesn’t support special characters).
I precise that I don’t use plugins.

Please, I’d like help, I’ve already spent a lot of times on it, and I can’t bear it anymore.
Let’s end this encoding problem on this post.

1 Like
2

Thank you a lot for your explanation and all of your links.
My “game” is a program to learn japanese in short, so it does have english too.
I’ve already tried L"", but no success with something like L"私" (I have to try L’\u79c1’).
I think I’ll follow your advice and stop trying to find something (for the moment), and I’ll check the Localization Dashboard since I’m unfamiliar with it.

Thank you again !

3

They not really “special characters”, since UE4 use UTF16 encoding to store strings (according to te doc below) they are byte encoded standard characters stored in 2 bytes as bytes, that the point of unicode encoding to make those character not special :slight_smile:

Problems you having relates to encoding of code files and issues related to those are explained here so read it up:

Note that at the end Epic themselves wrote that it does not recommend to use international string literals (which is what i assume you mean by “stocking”) in code files due to complications explained in there.

But this does not change the fact that strings store UTF16 data and you can input any character to text input in editor, this includes Blueprint where no limitations there on Strings (which is FString on C++ side). That means outside of issues related in C++ source files, entire engine works in UTF16 and support any character supported by that encoding seamlessly, so if you input Japanese character (with IME keyboard for example or copy paste) in editor or in game, engine handle that and UI (Slate/UMG) should show the character properly as long as you have font set up for them as well in as in C++ FString will store them properly and as long as you not operate in raw TCHAR you should not have issue operating those strings stored in FString on C++, FString internal code got you covered on that. Actually as programmer you should be fully aware that user can use any character on Slate/UMG made UI.

I also think you misunderstand what FText does, main difference from FString is it tied to UE4 localization system and it can seamlessly change text depending on language settings, it makes multilagual support a hell lot easier and allows to type in international strings in editor. Read about it here:

https://forums.unrealengine.com/development-discussion/content-creation/34379-localization-dashboard-preview-and-explanation-of-ue4-s-text-localization-process

So my recommendation is, you not getting all this encoding stuff… just use localization system and don’t type Japanese directly in to C++ files, it’s the safest way. Make your game English by default and use LOCTEXT and NSLOCTEXT to input any stuff that will be in japanese then use localization system which is explaned in first link to set up Japanese version localizationof the game.

In fact it is very recommended to use localization system on very begining of your project, even if you not thinking of translating the game to other languages, because you don’t know if in the future you will want to translate game to other languages. If you gonna change your mind and decide to translate you will find it too hard to do without localization system.

Alternatively if your aim is to make mixed Language game like having Japanese and English text in same time (as you didn’t explain what you really want to do with that Japanese text in C++) use blueprints to input Japanese text directly in, you can also probably find API which allows you to get text from localization system, wntire system is contained in single module here, you can access it via Get() static function (FInternationalization::Get()) from anywhere:

If you really need to input japanese character in C++ for text processing try using C++ Unicode escape symbols or L"":

https://en.cppreference.com/w/cpp/language/escape

But not sure if they gonna work

4

Ah i see, so you will need to interact with japanese characters in C++. then try to use escape code :slight_smile: if not you can always process raw TCHAR data as bytes if it’s really needed. You can also try to move some things to blueprint.

5

Yeah, I’ll try that, and that’s not a big deal if I can’t do it in C++, it’s just that I started to learn ue4 c++ a little time ago, so I wanted to learn more.
I forgot to answer you about it, but blueprints do work.