Can't run UnrealEditor with SDL_VIDEODRIVER=wayland

Net-yaroze · August 15, 2023, 7:20am

Heya,

I’m having issues with the UnrealEditor scaling with X11, so I thought I’d set SDL_VIDEODRIVER=wayland to see if that fixes any issues.

Unfortunately it does not work:

[2023.08.14-23.28.17:024][  0]LogInit: Initializing SDL.
[2023.08.14-23.28.17:024][  0]LogInit: Warning: Could not initialize SDL: wayland not available
[2023.08.14-23.28.17:024][  0]LogInit:  - Physical RAM available (not considering process quota): 63 GB (64194 MB, 65735120 KB, 67312762880 bytes)
[2023.08.14-23.28.17:024][  0]LogInit:  - VirtualMemoryAllocator pools will grow at scale 1.4
[2023.08.14-23.28.17:024][  0]LogInit:  - MemoryRangeDecommit() will be a no-op (re-run with -vmapoolevict to change)
[2023.08.14-23.28.17:024][  0]LogInit:  - PageSize 4096
[2023.08.14-23.28.17:024][  0]LogInit:  - BinnedPageSize 65536
[2023.08.14-23.28.17:024][  0]LogCsvProfiler: Display: Metadata set : extradevelopmentmemorymb="0"
[2023.08.14-23.28.17:029][  0]LogInit: Physics initialised using underlying interface: Chaos
[2023.08.14-23.28.17:042][  0]LogInit: Using OS detected language (en-GB).
[2023.08.14-23.28.17:042][  0]LogInit: Using OS detected locale (en-GB).
[2023.08.14-23.28.17:043][  0]LogTextLocalizationManager: No specific localization for 'en-GB' exists, so 'en' will be used for the language.
[2023.08.14-23.28.17:160][  0]LogInit: Initializing SDL.
[2023.08.14-23.28.17:160][  0]LogInit: Warning: Could not initialize SDL: wayland not available
[2023.08.14-23.28.17:160][  0]LogInit: Warning: FLinuxSplashState::InitSplashResources() : InitSDL() failed, there will be no splash.
[2023.08.14-23.28.17:160][  0]LogInit: Initializing SDL.
[2023.08.14-23.28.17:160][  0]LogInit: Warning: Could not initialize SDL: wayland not available
[2023.08.14-23.28.17:160][  0]LogInit: Error: FLinuxApplication::CreateLinuxApplication() : InitSDL() failed, cannot create application instance.
[2023.08.14-23.28.17:160][  0]LogCore: FUnixPlatformMisc::RequestExit(bForce=true, ReturnCode=1)
[2023.08.14-23.28.17:160][  0]LogCore: FUnixPlatformMisc::RequestExit(1)

I am assuming it is linked against different binaries?

cybin · October 22, 2023, 9:37pm

I’m stuck with this problem too. SDL2 is in ThirdParty/ and is probably built without Wayland support, unfortunately. If there is a way to compile it with all available drivers, it would be great. Or alternatively use the system library instead.

For now, I cannot use the editor in X11 due to redrawing problems and neither can run it on Wayland.

Any help is appreciated, thanks.

cybin · October 24, 2023, 3:00pm

Meanwhile I’ve been able to rebuild SDL2 with Wayland support. I’ve edited

UnrealEngine-5.3.1-release/Engine/Source/ThirdParty/SDL2/build.sh

and added -DSDL_WAYLAND=ON to the three lines at the bottom. Then I could run

UnrealEngine-5.3.1-release/Engine/Build/BatchFiles/Linux/BuildThirdParty.sh -b SDL2

to rebuild SDL2. I got these new builds in UnrealEngine-5.3.1-release/Engine/Source/ThirdParty/SDL2:

nm build-SDL-gui-backend-Debug/libSDL2d.a | grep Wayland | wc -l --> 204
nm build-SDL-gui-backend-Release/libSDL2.a | grep Wayland | wc -l --> 195
nm build-SDL-gui-backend-ReleasePIC/libSDL2.a | grep Wayland | wc -l --> 195

So, there are Wayland symbols available. But what still don’t want to work is to rebuild UnrealEditor and its ApplicationCore to use these ones.

But… there is also this directory tree:

SDL-gui-backend/lib
├── Linux
│  └── x86_64-unknown-linux-gnu
│     ├── libSDL2.a
│     ├── libSDL2_fPIC.a
│     └── libSDL2_fPIC_Debug.a
└── Unix
   ├── aarch64-unknown-linux-gnueabi
   │  ├── libSDL2.a
   │  ├── libSDL2_fPIC.a
   │  └── libSDL2_fPIC_Debug.a
   └── x86_64-unknown-linux-gnu
      ├── libSDL2.a
      ├── libSDL2_fPIC.a
      └── libSDL2_fPIC_Debug.a

nm SDL-gui-backend/lib/Linux/x86_64-unknown-linux-gnu/libSDL2.a | grep Wayland | wc -l --> 195
nm SDL-gui-backend/lib/Unix/x86_64-unknown-linux-gnu/libSDL2.a | grep Wayland | wc -l --> 0

I assume that it is linking UnrealEditor (or the module) with the Unix/ variant. Unfortunately, I could not get ./Build.sh to be verbose or veryverbose to check what is
happening in detail.

My questions are now…

How do I rebuild UnrealEditor, using one of my rebuilt libSDL2.a?
How do I clean the build that I can rebuild everything, alternatively?
How do I run UnrealBuildTool to be verbose to, at least, check what is going on?

Thanks in advance.

All-PurposeMat · November 7, 2023, 9:12am

I got as far as you did following your instructions. I can follow the steps to get a SDL .a build with Wayland support. However, running the engine (even after fully rebuilding it) does indeed not use our Wayland-enabled SDL2, as it still fails to run. I can force it to use my system SDL2 lib by defining SDL_DYNAMIC_API=/usr/share/libSDL2.so, but it crashes shortly after opening the initial project list window. My attempt to get it working is documented here: Mat: "@PJB thanks! This does get it to run on Wayland, …" - Gamedev Mastodon

mattgattis · April 28, 2024, 9:09pm

SDL2 is prebuilt and downloaded when you run Setup.sh. The file you need to rebuild is:

Engine/Source/ThirdParty/SDL2/SDL-gui-backend/lib/Unix/x86_64-unknown-linux-gnu/libSDL2_fPIC.a

I could not build this with the build.sh script in the SDL2 directory, as my system stdlib wont link with the SDK’s version. The only way I could build it was by running the RunMe.sh script in SDL2/docker, which automatically copies it to the right directory when its donw. This also fails because the version of SDL2 in the tree requires a newer libwayland-client than CentOS 7 has. So, here are all the changes I had to make:

RunMe.sh:

+BuildSDL2WithDocker x86_64-unknown-linux-gnu      centos:8
-BuildSDL2WithDocker x86_64-unknown-linux-gnu      centos:7

docker-build-sdl2.sh:

-if [ $UID -eq 0 ]; then
-  # Centos 7
-  yum install -y epel-release
-  yum install -y cmake3 make gcc-c++
-  yum install -y libXcursor-devel libXinerama-devel libxi-dev libXrandr-devel libXScrnSaver-devel libXi-devel mesa-libGL-devel mesa-libEGL-deve
l pulseaudio-libs-devel wayland-protocols-devel wayland-devel libxkbcommon-devel mesa-libwayland-egl-devel alsa-lib-devel libudev-devel
-  # Create non-privileged user and workspace
-  adduser buildmaster
-  mkdir -p /build
-  chown buildmaster:nobody -R /build
-  cd /build
-  exec su buildmaster "$0"
-fi

+ulimit -n 1024000
+cd /etc/yum.repos.d/
+sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
+sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
+yum update -y
+yum install -y epel-release cmake make gcc-c++ libXcursor-devel libXinerama-devel libXi-devel libXrandr-devel libXScrnSaver-devel libXi-devel m
esa-libGL-devel mesa-libEGL-devel pulseaudio-libs-devel wayland-protocols-devel wayland-devel libxkbcommon-devel mesa-libwayland-egl-devel alsa-
lib-devel libudev-devel
+mkdir -p /build
+cd /build

...

 OPTS+=(-DCMAKE_C_FLAGS=-gdwarf-4)
+OPTS+=(-DSDL_WAYLAND=ON)

Then:

./RunMe.sh
verify you got new timestamps on the .a files in Engine/Source/ThirdParty/SDL2/SDL-gui-backend/lib/Unix/x86_64-unknown-linux-gnu/
in root unreal dir, “make CrashReportClient ShaderCompileWorker UnrealLightmass InterchangeWorker UnrealPak UnrealEditor”
SDL_VIDEODRIVER=wayland ./Engine/Binaries/Linux/UnrealEditor

Quite a pain but the end result is that wayland seems to work quite nicely. I’m running the NVIDIA proprietary driver on arch linux gnome wayland session, and running with xwayland was super glitchy for me. Now with native SDL wayland, it’s usable as far as I can tell.

Cestarian · May 9, 2024, 4:06am

I cannot figure out how to do this, cannot find this runme.sh file (maybe cuz i’m using the precompiled linux binaries? maybe a version difference?). I ended up just setting up i3 and switching between TTYs to run the editor lol.

I’ve submitted a bug report though (or tried to at least, i don’t see my report on their issue tracker so who knows), hopefully they fix it soon.