Demangled used std::__1::basic_string but I need std::__cxx11::basic_string

Development Platform & Builds
, , , , ,
1

Hi all,

I’m trying to use a Shared Library (.so) in a Unreal Project. I already have tried Unreal 4.14.x, 4.15.x. 4.16.x, 4.17.x, 4.18.0 and I always have one problem or another.

At the end, I came back to Unreal 4.14.3 and the problem that I’m having is that Unreal can’t find a symbol when I run my external_library::init.

With the name of the symbol raised on the error, I used c++filt SYMBOL_HERE to demangle.

The demangled symbol is:

external_library::init(int&, char**, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, unsigned int)

Using nm and c++filt I was able to demangle all symbols on the external_library, and the demangled version of external_libray::init found on the .so is:

external_library::init(int&, char**, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int)

If you use diff (or the graphical version called meld), you can see the Unreal is using std::__1::basic_string but the compiled .so library is using std::__cxx11::basic_string, which means c++11.

In order to try to solve this problem, I have changed LinuxToolChain.cs to pass the parameter -std=c++11 when compiling my Unreal Project, but this didn’t solve the problem, because Unreal keeps using std::__1::basic_string instead of std::__cxx11::basic_string.

Then my question is: how can I force unreal use c++11, specially for std::string?

I already have tried everything that I can but I couldn’t find a solution.

2

This is because your library is compiled against libstdc++, while Unreal by default uses bundled libc++. Different STL implementations decorate symbols differently to avoid clashes (which would be even worse).

Your options are:

  • compile your library against libc++ (which means using clang with -stdlib=libc++ argument)
  • switch LinuxToolChain.cs to NOT use libc++ (but that may mean rebuilding some other libraries bundled with Unreal).
  • Avoid using STL across module boundaries (have extern “C” interface for your library).

If you’re using UE earlier than 4.16, you’ll need to remove (in LinuxToolChain.cs) the linker script that hides new/delete signatures or you may run into other obscure problems.

1 Like
3

I have modified the LinuxToolChain.cs on Unreal 4.14.3 to disable libc++ (by chaning if (ShouldUseLibcxx(LinkEnvironment.Config.Target)) to if (ShouldUseLibcxx(LinkEnvironment.Config.Target) && false)) and build Unreal from zero two times, and there is a library that still using std::__1::basic_iostream instead of libstdc++.

The library that fails to load is Engine/Binaries/Linux/libUE4Editor-ShaderFormatOpenGL.so because the symbol _ZTINSt3__114basic_iostreamIcNS_11char_traitsIcEEEE is not found.

The symbol demangled is typeinfo for std::__1::basic_iostream<char, std::__1::char_traits<char> >

This libUE4Editor-ShaderFormatOpenGL.so should be compiled with libstdc++ since I have disabled libc++ in LinuxToolChain.cs, right?

What do you think I’m doing wrong now?

Thanks.

4

Nothing wrong, you just need to rebuild the third party libraries bundled with Unreal to use your system’s libstdc++. Most third party software provided with the engine is built against libc++ to avoid potential incompatibilities between libstdc++ on different systems (libstdc++ cannot be bundled due to its license).

EDIT: in this case I believe that rebuilding hlslcc should unblock you.

5

If I download the Unreal code, run ./Setup.sh, ./GenerateProjectFiles.sh and make, I building the entire Unreal and the Third Party as well,right? Because this is what I have done, but changing the LinuxToolChain.cs before running ./Setup.sh.

Or the ThirdParty are previously compiled and downloaded when I run ./Setup.sh?

If they are previously built, what are the steps to rebuild them? I only know how to rebuild my own unreal projects or the “entire Unreal” with the steps mentioned above.

6

No, Third Party libraries are supplied as pre-built binaries and aren’t rebuilt during the usual build process. The reasons for that is that some of them can take a long to compile (e.g. PhysX), unnecessarily making engine compilation even longer, while others require specialized environment/headers that aren’t available on most user systems out of the box (the third reason is that for some libraries - like steam or FBX sdk - there are no sources at all).

There is unfortunately no uniform rebuild process for rebuilding them. Each library in Engine/Source/ThirdParty contains some kind of a readme file or sometimes build scripts that can facilitate rebuilding, but may not work right away on your system.

In your case, you need to go to Engine/Source/ThirdParty/hlslcc/hlslcc/projects/Linux and change the Makefile to not use libc++, then run make and replace the resulting .a file with the locally built one

Arguably a better solution is not to use C++ interface (especially STL classes) when calling a third party DSO from Unreal, which would greatly reduce or likely eliminate binary compatibility issues altogether.

7

What I have done:

  1. Changed the Makefile you mentioned
  2. Compiled that specific ThirdParty
  3. Replaced all libhlslcc.a with the one that was generated locally.
  4. After that I went to the root folder of Unreal, executed ./GenerateProjectFiles.sh and later make.
  5. Then I’ve removed all temp files of my project (rm Binaries/ Content/StarterContent/ DerivedDataCache/ Intermediate/AssetRegistryCache/ Intermediate/Build/Linux/ Intermediate/CachedAssetRegistry.bin Intermediate/Config/ Intermediate/ReimportCache/ Saved/ -rf)
  6. Compiled by project again
  7. Crashed with the same error when trying to open the project.

I think that many of the problems that I’m facing is because I don’t understand very well how Unreal itself is structured.

On the documentation I can find instructions for users that want just want to develop games that work in “any” platform, but in my case, I want something else.

I don’t need all Unreal capabilities (like compile for Android, Windows, IOS, etc).

I would like to use Unreal only for Linux, the smallest possible version of Unreal, specially to avoid problems because of “incompatibilities”.

So my “final” question is: Is there any documentation about how Unreal itself is structured? How could I disable “Build for Android” or how completely disable “Blueprints” and only use C++, for example?

I couldn’t find (yet) this information on the documentation.

8

Before running in step 5, you may want to double check that your plugin is compiled against libstdc++ (i.e. all STL symbols in it are mangled appropriately). Turning off libc++ and switching to libstdc++ is not something that we officially support. UBT has code to use libstdc++ to make it easier for advanced folks who want to modify the engine, but we don’t test that path, so hlslcc may not be the only thing that needs to be changed.

Even on Linux there’s a problem of binary compatibility between different libstdc++ versions (for example between CentOS/RHEL 7 and Ubuntu 17.10), this is why we opted to bundle libc++ instead so the same UE4 binary can run on both.

Re: structuring of the engine modules, I am not sure if this is covered in documentation, which is geared towards gameplay programmers. Some concepts are discussed here: Programming with CPP in Unreal Engine | Unreal Engine 5.1 Documentation However, if you want to substantially modify the engine you will probably need to experiment first. You can absolutely use just C++ and not program in Blueprints, but this is a separate question from compiling out Blueprint modules. Reducing the number of modules to minimum takes some experimentation and modifications to the engine due to existing interdependencies.

9

Hi ,

I went through the steps again from the beginning on Unreal4.14.3, removed (in LinuxToolChain.cs) the part the hides the new/delete signatures, compiled HLSLCC with libstdc++
and that libc++/libstdc++ problem when loadinglibUE4Editor-ShaderFormatOpenGL.so was solved.

THANK YOU VERY MUCH FOR THE INSTRUCTIONS!!!

10

, I have created a new question about Large Memory Pool error on a computer with 60GB because trying to ask as a comment wasn’t possible (I don’t know why).

Could you have a look at the question? I already tried to solve it looking for similar problems on the Unreal forum but I couldn’t make it work.

Thank you very much.