ARCore Camera Feed Issue - UE 5.2.0

Has someone tried if this is resolved in 5.2.1 hotfix? Or found a cause/solution for the issue in 5.2?

5.2.1 didnt fix it. M_PassthroughCamera material is indeed the problem as someone suggested above. More specifically, the Map AR Passthrough Camera UVs.
image

Just tried to build it using 5.2.1 hotfix same problem…

Anyone knows how could we fix the material M_PassthroughCamera?

This is what I see when I start an AR session on 5.2.1:

I’m hitting this issue as well, on 5.2.1. I tested it on both MacOS (M2) and Windows.

On MacOS, the shader compilation fails and there is no stack trace or useful error message, even with shader debugging turned on, it just says “Internal Error!” on the AR material compile steps.

On Windows, I do get a stack trace, and it looks like it is complaining about a missing function on the std::vector class when doing the optimization pass over the SPIRV shader. Note that several materials fail - I think the shared function in the shader is the camera UV lookup function mentioned earlier.

LogPlayLevel: Warning: UAT: LogInit: Display: LogShaderCompilers: Warning: Failed to compile Material /GoogleARCore/M_SceneDepthOcclusion.M_SceneDepthOcclusion for platform GLSL_ES3_1_ANDROID, Default Material will be used in game.
LogPlayLevel: UAT: Exception encountered in platform compiler: Exception: CPP_EXCEPTION, address=0x000000007732536c
LogPlayLevel: UAT:
LogPlayLevel: UAT: Exception Callstack:
LogPlayLevel: UAT: 0x00007ffa7732536c KERNELBASE.dll!UnknownFunction []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff9bae86720 VCRUNTIME140.dll!UnknownFunction []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff9ba50b762 MSVCP140.dll!UnknownFunction []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff928180b21 ShaderConductor.dll!std::vector<wchar_t const *,std::allocator<wchar_t const *> >::_Xlength() [C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.34.31933\include\vector:2201]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff92830b49d ShaderConductor.dll!spvtools::MakeUnique<std::vector<unsigned int,std::allocator >,std::vector<unsigned int,std::allocator > &>() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\External\SPIRV-Tools\source\util\make_unique.h:25]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff92830bcc5 ShaderConductor.dll!spvtools::utils::SmallVector<unsigned int,2>::operator=() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\External\SPIRV-Tools\source\util\small_vector.h:108]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff928332cc2 ShaderConductor.dll!<lambda_51198c386a817a2d673755079076ea5b>::operator()() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\External\SPIRV-Tools\source\opt\reduce_const_array_to_struct_pass.cpp:209]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff9283ba904 ShaderConductor.dll!std::_Func_impl_no_alloc<<lambda_f550882f3894c785da4fc1b0c46aead2>,bool,spvtools::opt::Instruction *>::_Do_call() [C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.34.31933\include\functional:833]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff9283cc901 ShaderConductor.dll!spvtools::opt::analysis::DefUseManager::WhileEachUser() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\External\SPIRV-Tools\source\opt\def_use_manager.cpp:123]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff9283cc248 ShaderConductor.dll!spvtools::opt::analysis::DefUseManager::ForEachUser() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\External\SPIRV-Tools\source\opt\def_use_manager.cpp:135]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff928333eb5 ShaderConductor.dll!spvtools::opt::ReduceConstArrayToStructPass::ReduceArray() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\External\SPIRV-Tools\source\opt\reduce_const_array_to_struct_pass.cpp:194]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff9283338f9 ShaderConductor.dll!spvtools::opt::ReduceConstArrayToStructPass::Process() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\External\SPIRV-Tools\source\opt\reduce_const_array_to_struct_pass.cpp:79]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff92830db26 ShaderConductor.dll!spvtools::opt::Pass::Run() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\External\SPIRV-Tools\source\opt\pass.cpp:38]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff92831745f ShaderConductor.dll!spvtools::opt::PassManager::Run() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\External\SPIRV-Tools\source\opt\pass_manager.cpp:59]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff928300c19 ShaderConductor.dll!spvtools::Optimizer::Run() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\External\SPIRV-Tools\source\opt\optimizer.cpp:642]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff92817cb1b ShaderConductor.dll!ShaderConductor::Compiler::Optimize() [D:\5.2\Engine\Source\ThirdParty\ShaderConductor\ShaderConductor\Source\Core\ShaderConductor.cpp:1968]
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ffa534764a5 ShaderCompileWorker-ShaderCompilerCommon.dll!CrossCompiler::FShaderConductorContext::OptimizeSpirv() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff8ff027a67 ShaderCompileWorker-ShaderFormatOpenGL.dll!CompileToGlslWithShaderConductor() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff8ff02664a ShaderCompileWorker-ShaderFormatOpenGL.dll!FOpenGLFrontend::CompileShader() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff8ff027088 ShaderCompileWorker-ShaderFormatOpenGL.dll!FShaderFormatGLSL::CompileShader() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff8feb2b88b ShaderCompileWorker-RenderCore.dll!CompileShaderInternal() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff8feb2b612 ShaderCompileWorker-RenderCore.dll!CompileShader() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff605c2d64e ShaderCompileWorker.exe!FWorkLoop::ProcessInputFromArchive() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff605c282a2 ShaderCompileWorker.exe!FWorkLoop::Loop() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff605c2590a ShaderCompileWorker.exe!GuardedMain() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff605c25b74 ShaderCompileWorker.exe!GuardedMainWrapper() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff605c36ea8 ShaderCompileWorker.exe!wmain() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ff605c3861c ShaderCompileWorker.exe!__scrt_common_main_seh() []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ffa77d226ad KERNEL32.DLL!UnknownFunction []
LogPlayLevel: UAT:
LogPlayLevel: UAT: 0x00007ffa79d2a9f8 ntdll.dll!UnknownFunction []
LogPlayLevel: UAT:

Having done some digging, I suspect the bug is related to this commit: https://github.com/EpicGames/UnrealEngine/commit/3e13bde4ddf3f90cc0d91cf4508a9c48dff64561

The expression that I think the optimizer is struggling with is accessing the XRPassthroughCameraUVs Vector4, and this commit landed after 5.1 and altered the metadata around this uniform.

Unfortunately I don’t see a way to work around it in the plugin itself, which I’ve forked. But if I figure out a workaround I’ll post it here.

1 Like

Hi everyone! I’m also having this texture issue whenever I deploy the template. Has anyone found a solution?

Hello! :slight_smile:

It seems like the problem is the core functionality of the materials. The comments from @gfodor are not a solution but point to the problem.

From my side - me and my team still use the 5.0.3 production version as it is more/less stable. And I don’t believe we will have any attempts to fix the issue - only checking the newer versions along they will be coming.

1 Like

Thank you!

1 Like

I have tried several times registering a bug submission on this issue, but it never seems to register in their list of issues. Not sure why. Did someone else sucessfully address this issue to the developers?

I am not able to downgrade my project since google in-app purchases requires a newer version of unreal for its minimal allowed API version. Currently thinking of rebuilding the entire game in unreal 5.0.3 but that is lots of work to be done and is not very fun thing to do.

We’ve done a bunch of work on ARCore in 5.3 (including fixing this issue). It’s already available on GitHub if you’re comfortable working with source.

1 Like

This was fixed in UE5/Main here: https://github.com/EpicGames/UnrealEngine/commit/c0d2a35b7b35eec2e0b0e2c72821803b04b9f0a6
The ShaderConductor binaries were updated to latch the fix here: https://github.com/EpicGames/UnrealEngine/commit/08b9ae2ac901d1ade3a5bc1a04e3d1e8f852f493

The key change here is the update to Commit.gitdeps.xml, which tells our engine setup process (driven by Setup.bat) where to fetch our binary blobs.

If you want to migrate the fix into your local tree, you have a few options (besides switching to 5.3)

  • Switch to 5.3, grab the ShaderConductor binaries, and copy them back to 5.2
  • Stay in 5.2, modify Commit.gitdeps.xml to match 5.3 (or this commit) and re-run Setup.bat.
  • Migrate the ShaderConductor code fix into your 5.2 tree, and rebuild ShaderConductor locally
2 Likes

I implemented the 5.3 ShaderConductor binaries with the 5.2 build, but found on deploy that the camera feed was mirrored. To account for this, I started messing with the passthrough UVs in the screen space pass through shader but the feed is still getting mirrored. any ideas on how to resolve this? or did i grab out-dated binaries from the 5.3 preview?

1 Like

Hello,

Yesterday I got info from one of my colleagues that he was able to launch the 5.2.1 with the solution described above by EngineDevRob.

There are steps he mentioned:

TODO to fix the problem in 5.2

1. Download Unreal Engine 5.3 from zip
2. Copy ShaderConductor directory from \UE_5.3\Engine\Binaries\ThirdParty
3. Paste the directory in the same place in EU5.2

I can also share the file if you need it. I haven’t tried the solution yet but for him, it works without issues. At least for now.

UPD:
I tried the solution and it does work. Our team currently work on the 5.2.1 production version of the engine

2 Likes

Actually, I can say that it works for me. For those who do not want to download the entire 5.3 version, here is the folder you are looking for : ShaderConductor

As said before, go to …/UE_5.2/Engine/Binaries/ThirdParty, remove the old ShaderConductor folder and paste the new one. Then upload your project to your Android phone and the camera should work

6 Likes

Even after migrating to 5.3, I was still having the pass through camera mirroring problem, so for anyone else that has an issue with this, the following math can be applied to the pass through camera shader (use the simple math for reference, not the output in the example which is for viewport cams):
Mirror Camera Math

That being said, did anyone else have this issue? I’m testing on an Galaxy S8+ which is like close to min-spec for AR these days.

Anyhow, thanks for hooking up the ShaderConductor libraries for this fix!

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.