Editor crash by setting viewport to orthographic (Vulkan,Linux)

Development Rendering
, , , ,
1

Hello dear fellow UE devs,

I’m using Unreal for my 2d sidescroller since ~4.20, currently using 5.0.2. Had to switch my nvidia gpu for a amd one and now that I’m able to use vulkan as a rendering backend I’m experiencing a lot of editor crashes when switching to orthographic view.

Here are a few examples:

  1. Switching from “Perspective” to “Right” mode (Right being the default for Paper2D, as opposed to front)
  2. Clicking on my orthographic camera actor (Must be because of that little preview window popping up)
  3. Playing in Fullscreen (Non-Fullscreen works with my ortho camera for some reason)

Setupwise, I’m using some rather exotic stuff, which wasnt a problem using OpenGL:

  • Gentoo Linux
  • amd rx6650 gpu
  • Wayland (SwayWM, wlroots)
  • custom Kernel
  • a DIY apparmor profile for Unreal (though switching that off didnt help)
  • UE5 built from source (vanilla 5.0.2, no modification)
  • Vulkan (duh!)

The crash log tends looks like this:

Assertion failed: !EnumHasAnyFlags(StencilAccess, AnyCopy) [File:./Runtime/VulkanRHI/Private/VulkanBarriers.cpp] [Line: 439] Only the Stencil being transitioned to Copy state, this is not supported.

libUnrealEditor-VulkanRHI.so!GetDepthStencilStageAndAccessFlags(ERHIAccess, ERHIAccess, unsigned int&, unsigned int&, VkImageLayout&, bool) [/home/user/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanBarriers.cpp:439]
libUnrealEditor-VulkanRHI.so!FVulkanDynamicRHI::RHICreateTransition(FRHITransition*, FRHITransitionCreateInfo const&) [/home/user/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanBarriers.cpp:800]
libUnrealEditor-RenderCore.so!FRDGBarrierBatchBegin::CreateTransition() [/home/user/UnrealEngine/Engine/Source/./Runtime/RenderCore/Private/RenderGraphPass.cpp:162]
libUnrealEditor-RenderCore.so!FRDGBuilder::CreatePassBarriers() [/home/user/UnrealEngine/Engine/Source/./Runtime/RenderCore/Private/RenderGraphBuilder.cpp:2550]
libUnrealEditor-Core.so!TGraphTask<TFunctionGraphTaskImpl<void (ENamedThreads::Type, TRefCountPtr<FGraphEvent> const&), (ESubsequentsMode::Type)0> >::ExecuteTask(TArray<FBaseGraphTask*, TSizedDefaultAllocator<32> >&, ENamedThreads::Type, bool) [/home/user/UnrealEngine/Engine/Source/Runtime/Core/Public/Async/TaskGraphInterfaces.h:975]
libUnrealEditor-Core.so!void LowLevelTasks::FTask::Init<FTaskGraphCompatibilityImplementation::QueueTask(FBaseGraphTask*, bool, ENamedThreads::Type, ENamedThreads::Type)::'lambda'()>(char16_t const*, LowLevelTasks::ETaskPriority, FTaskGraphCompatibilityImplementation::QueueTask(FBaseGraphTask*, bool, ENamedThreads::Type, ENamedThreads::Type)::'lambda'()&&, bool)::'lambda'(bool)::operator()(bool) const [/home/user/UnrealEngine/Engine/Source/Runtime/Core/Public/Async/Fundamental/Task.h:350]
libUnrealEditor-Core.so!LowLevelTasks::TTaskDelegate<void (bool), 48u>::TTaskDelegateImpl<void LowLevelTasks::FTask::Init<FTaskGraphCompatibilityImplementation::QueueTask(FBaseGraphTask*, bool, ENamedThreads::Type, ENamedThreads::Type)::'lambda'()>(char16_t const*, LowLevelTasks::ETaskPriority, FTaskGraphCompatibilityImplementation::QueueTask(FBaseGraphTask*, bool, ENamedThreads::Type, ENamedThreads::Type)::'lambda'()&&, bool)::'lambda'(bool), false>::CallAndMove(LowLevelTasks::TTaskDelegate<void (bool), 48u>&, void*, unsigned int, bool) [/home/user/UnrealEngine/Engine/Source/Runtime/Core/Public/Async/Fundamental/TaskDelegate.h:171]
libUnrealEditor-Core.so!bool LowLevelTasks::FScheduler::TryExecuteTaskFrom<&(LowLevelTasks::TLocalQueueRegistry<1024u>::TLocalQueue::DequeueGlobal(bool, bool)), false>(LowLevelTasks::TLocalQueueRegistry<1024u>::TLocalQueue*, LowLevelTasks::TLocalQueueRegistry<1024u>::FOutOfWork&, bool, bool) [/home/user/UnrealEngine/Engine/Source/./Runtime/Core/Private/Async/Fundamental/Scheduler.cpp:278]
libUnrealEditor-Core.so!LowLevelTasks::FScheduler::WorkerMain(LowLevelTasks::FSleepEvent*, LowLevelTasks::TLocalQueueRegistry<1024u>::TLocalQueue*, unsigned int, bool) [/home/user/UnrealEngine/Engine/Source/./Runtime/Core/Private/Async/Fundamental/Scheduler.cpp:320]
libUnrealEditor-Core.so!FThreadImpl::Run() [/home/user/UnrealEngine/Engine/Source/./Runtime/Core/Private/HAL/Thread.cpp:66]
libUnrealEditor-Core.so!FRunnableThreadPThread::Run() [/home/user/UnrealEngine/Engine/Source/./Runtime/Core/Private/HAL/PThreadRunnableThread.cpp:25]
libUnrealEditor-Core.so!FRunnableThreadPThread::_ThreadProc(void*) [/home/user/UnrealEngine/Engine/Source/Runtime/Core/Private/HAL/PThreadRunnableThread.h:185]
libc.so.6!UnknownFunction(0x87b26)
libc.so.6!UnknownFunction(0x109ffb

Anyone got any idea how to fix this? I’m absolutely clueless about SUCH low-levely stuff.
Guess I’m gonna file a bug right after this…

Any help is appreciated!

Greez
Vinni

3 Likes
3

Hi.

I’m running through the same issue with some standard equipment (Arch Linux, 3080ti). I don’t quite understand the patch you did. it looks like you removed and pasted the same thing? i’m not sure, can you help me?

4

Okay, I guess commenting-out assertion checks isn’t the greatest of ideas, but in my case it did the trick and everything works now. Still going to file a bug, but for anyone needing that changed right now, here’s a patch for that:

--- a/Engine/Source/Runtime/VulkanRHI/Private/VulkanBarriers.cpp
+++ b/Engine/Source/Runtime/VulkanRHI/Private/VulkanBarriers.cpp
@@ -436,7 +436,7 @@ static void GetDepthStencilStageAndAccessFlags(ERHIAccess DepthAccess, ERHIAcces
    }
    else
    {
-       checkf(!EnumHasAnyFlags(StencilAccess, AnyCopy), TEXT("Only the Stencil being transitioned to Copy state, this is not supported."));
+       //checkf(!EnumHasAnyFlags(StencilAccess, AnyCopy), TEXT("Only the Stencil being transitioned to Copy state, this is not supported."));
    }
}

Edit: Forgot to comment out the line in question in the patch, which lead to confusion of some. Also had I marked this as the solution already, so I couldn’t edit it anymore I suppose. That’s why this is the comment of a deleted post…

3 Likes
5

Hello Jarmanitor,
thanks for pointing out that my patch did nothing! :smiley:
Corrected that and now it actually comments-out the line, which was giving us a headache.
(Patching that obviously unnescessarily complicated things, just get rid of that line and you should be golden!)
Hope everything makes sense now.
Greez
Vinni

6

I had this same problem (Ubuntu 20.04 with Nvidia 515 driver and vulkan 1.2.131)…commenting out that line and re-compiling worked to fix it…

However I think it may cause some other issues. When repositioning objects in 2D view the emissive lighting materials get messed up. After repositioning the emissive material actor in 2D view the lighting is broken; it does not cast light correctly (at certain angles the light disappears). If you reposition the actor in normal 3D perspective mode then this issue does not occur.

7

I have this issue but where do i need to comment that line? Where is the file?

8

Thanks after commenting and compiling it works like a charm

9

I have the very same issue here (Arch, 3080Ti), but 5.0.3. So bug is not fixed, do you know where to report it ?

10

same on windows , rti 3080 , vulkan

11

Seems fixed on version 5.1.0 preview 2

I downloaded the Linux_Unreal_Engine_5.1.0_preview-2.zip
And the same scene which crashed when going to top view now works.

2 Likes
12

Can you check bugs that happened to me are gone in 5.1?
try these:
-Orthographic views
-Wireframe view mode
-Maximize Viewport button

i have no time to download the latest zip file for now.

13

Same with me on Arch Linux Kernel 6.0.6. UE5 5.0.3.
It crashes in many things:
-Orthographic views
-Wireframe view mode
-Maximize Viewport button
-and many more i guess …

It’s almost unusable in Linux. I can’t learn too far, bcs I will get many crashes while following tutorials xD