How to use "AR Session" and "Add Runtime Candidate Image" with Blueprint ?

SilvRO · April 11, 2023, 7:48am

Good morning,

I’m trying to dynamically add datasets to ARSessionConfig. My goal is to add candidate images from a DataTable without manually populating the session from the editor.

According to my debug traces, the candidate images are indeed loaded in the session. I debug this with Get AR Session > Get Candidate Image List > Foreach > Print String. But there is no detection, except on the candidate images entered manually in ARSessionConfig. I use Debug Draw Tracked Geometry on each tracked images.

The documentation for Add Runtime Candidate Image says to restart the AR session. What I do (I think). After loading all the candidate images, I make a Delay of 0.5 seconds and I execute Stop AR Session then Start AR Session (with or without Delay between, with or without Stop AR Session, …).

Adding candidate images is done from Event Begin Play of the ARPawn.

To date, I have not found any clear documentation/example on this subject.

Anyone have an idea where the problem could come from? Or give me a working example?

thanks in advance

note : I can provide screenshots of Blueprints if needed

SilvRO · April 13, 2023, 9:06am

I’ve some errors when Add Runtime Candidate Image is executed. This bloc of log is display on itch iteration but not alway (when it deployed on Android, the errors are random):

LogPlayLevel: Warning:
LogOutputDevice: Warning:

Script Stack (0 frames) :

Ensure condition failed: CurrentGCObject->GetReferencerName() != FGCObject::UnknownGCObjectName [File:D:/build/++UE5/Sync/Engine/Source/Runtime/CoreUObject/Private/Misc/GCObjectReferencer.cpp] [Line: 168]
Please make sure all FGCObject derived classes have a unique name by overriding FGCObject::GetReferencerName() function. FGCObject::GetReferencerName() will become pure virtual in the next engine release. See callstack for details.
LogStats: FPlatformStackWalk::StackWalkAndDump -  0.078 s

=== Handled ensure: ===
LogPlayLevel:
Ensure condition failed: !bIsInlined || BulkMeta.GetOffset() == INDEX_NONE || IgnoreInlineDataReloadEnsures.bEnabled  [File:D:/build/++UE5/Sync/Engine/Source/Runtime/CoreUObject/Private/Serialization/BulkDataStreaming.cpp] [Line: 128]
Attempting to stream inline BulkData! This operation is not supported by the IoDispatcher and so will eventually stop working. The calling code should be fixed to retain the inline data in memory and re-use it rather than discard it and then try to reload from disk!
Stack:
0x00000071E35F3C9C libUnreal.so(0x000000000D60DC9C)!UE::BulkData::Private::EnsureCanStreamBulkData(UE::BulkData::Private::FBulkMetaData const&)  []
0x00000071E35F55E0 libUnreal.so(0x000000000D60F5E0)!UE::BulkData::Private::OpenReadBulkData(UE::BulkData::Private::FBulkMetaData const&, UE::BulkData::Private::FBulkDataChunkId const&, long long, long long, EAsyncIOPriorityAndFlags, TFunction<void (FArchive&)>&&)  []
0x00000071E35CF4B0 libUnreal.so(0x000000000D5E94B0)!FBulkData::TryLoadDataIntoMemory(FIoBuffer)  []
0x00000071E35CF940 libUnreal.so(0x000000000D5E9940)!FBulkData::MakeSureBulkDataIsLoaded()  []
0x00000071E35CF7AC libUnreal.so(0x000000000D5E97AC)!FBulkData::Lock(unsigned int)  []
0x00000071E057FA4C libUnreal.so(0x000000000A599A4C)!FGoogleARCoreXRTrackingSystem::OnAddRuntimeCandidateImage(UARSessionConfig*, UTexture2D*, FString, float)  []
0x00000071E53D8598 libUnreal.so(0x000000000F3F2598)!FARSupportInterface::AddRuntimeCandidateImage(UARSessionConfig*, UTexture2D*, FString, float)  []
0x00000071E53CC9EC libUnreal.so(0x000000000F3E69EC)!UARBlueprintLibrary::AddRuntimeCandidateImage(UARSessionConfig*, UTexture2D*, FString, float)  []
0x00000071E53CC7F4 libUnreal.so(0x000000000F3E67F4)!UARBlueprintLibrary::execAddRuntimeCandidateImage(UObject*, FFrame&, void*)  []
0x00000071E38BAD34 libUnreal.so(0x000000000D8D4D34)!UObject::execCallMathFunction(UObject*, FFrame&, void*)  []
0x00000071E38C8DA0 libUnreal.so(0x000000000D8E2DA0)!UObject::execLetObj(UObject*, FFrame&, void*)  []
0x00000071E38BD4F0 libUnreal.so(0x000000000D8D74F0)!ProcessLocalScriptFunction(UObject*, FFrame&, void*)  []
0x00000071E38BD080 libUnreal.so(0x000000000D8D7080)!UObject::ProcessInternal(UObject*, FFrame&, void*)  []
0x00000071E362FC08 libUnreal.so(0x000000000D649C08)!UFunction::Invoke(UObject*, FFrame&, void*)  []
0x00000071E38C2C58 libUnreal.so(0x000000000D8DCC58)!UObject::ProcessEvent(UFunction*, void*)  []
0x00000071E752B1B4 libUnreal.so(0x00000000115451B4)!AActor::ProcessEvent(UFunction*, void*)  []
0x00000071E7FE45D0 libUnreal.so(0x0000000011FFE5D0)!FLatentActionManager::TickLatentActionForObject(float, TMultiMap<int, FPendingLatentAction*, FDefaultSetAllocator, TDefaultMapHashableKeyFuncs<int, FPendingLatentAction*, true> >&, UObject*)  []
0x00000071E7FE3EFC libUnreal.so(0x0000000011FFDEFC)!FLatentActionManager::ProcessLatentActions(UObject*, float)  []
0x00000071E752C4AC libUnreal.so(0x00000000115464AC)!AActor::Tick(float)  []
0x00000071E7527A20 libUnreal.so(0x0000000011541A20)!FActorTickFunction::ExecuteTick(float, ELevelTick, ENamedThreads::Type, TRefCountPtr<FGraphEvent> const&)  []
0x00000071E8AF6D14 libUnreal.so(0x0000000012B10D14)!FTickFunctionTask::DoTask(ENamedThreads::Type, TRefCountPtr<FGraphEvent> const&)  []
0x00000071E8AF66F0 libUnreal.so(0x0000000012B106F0)!TGraphTask<FTickFunctionTask>::ExecuteTask(TArray<FBaseGraphTask*, TSizedDefaultAllocator<32> >&, ENamedThreads::Type, bool)  []
0x00000071E2D39D4C libUnreal.so(0x000000000CD53D4C)!FNamedTaskThread::ProcessTasksNamedThread(int, bool)  []
0x00000071E2D38008 libUnreal.so(0x000000000CD52008)!FNamedTaskThread::ProcessTasksUntilQuit(int)  []
0x00000071E2D370C8 libUnreal.so(0x000000000CD510C8)!FTaskGraphCompatibilityImplementation::ProcessThreadUntilRequestReturn(ENamedThreads::Type)  []
0x00000071E2D37660 libUnreal.so(0x000000000CD51660)!FTaskGraphCompatibilityImplementation::WaitUntilTasksComplete(TArray<TRefCountPtr<FGraphEvent>, TSizedInlineAllocator<4u, 32, TSizedDefaultAllocator<32> > > const&, ENamedThreads::Type)  []
0x00000071E8AEEE08 libUnreal.so(0x0000000012B08E08)!FTickTaskSequencer::ReleaseTickGroup(ETickingGroup, bool)  []
0x00000071E8AE9D4C libUnreal.so(0x0000000012B03D4C)!FTickTaskManager::RunTickGroup(ETickingGroup, bool)  []
0x00000071E8046340 libUnreal.so(0x0000000012060340)!UWorld::Tick(ELevelTick, float)  []
0x00000071E7D68F0C libUnreal.so(0x0000000011D82F0C)!UGameEngine::Tick(float, bool)  []
0x00000071E050E510 libUnreal.so(0x000000000A528510)!FEngineLoop::Tick()  []
0x00000071E0506EFC libUnreal.so(0x000000000A520EFC)!AndroidMain(android_app*)  []
0x00000071E0515448 libUnreal.so(0x000000000A52F448)!android_main()  []
0x00000071E054C380 libUnreal.so(0x000000000A566380)![Unknown]()  []
0x00000072DF839F24 libc.so(0x00000000000E6F24)![Unknown]()  []
0x00000072DF7D80CC libc.so(0x00000000000850CC)![Unknown]()  []This text will be hidden

SilvRO · April 13, 2023, 2:58pm

And an other issue that is always done :

Script Stack (1 frames) :
/Game/HandheldAR/Blueprints/GameFramework/BP_ARPawn.BP_ARPawn_C.ExecuteUbergraph_BP_ARPawn

Ensure condition failed: !bIsInlined || BulkMeta.GetOffset() == INDEX_NONE || IgnoreInlineDataReloadEnsures.bEnabled [File:D:/build/++UE5/Sync/Engine/Source/Runtime/CoreUObject/Private/Serialization/BulkDataStreaming.cpp] [Line: 128]
Attempting to stream inline BulkData! This operation is not supported by the IoDispatcher and so will eventually stop working. The calling code should be fixed to retain the inline data in memory and re-use it rather than discard it and then try to reload from disk!
LogStats: FPlatformStackWalk::StackWalkAndDump -  0.242 s
LogOutputDevice: Error: === Handled ensure: ===
LogOutputDevice: Error:
LogOutputDevice: Error: Ensure condition failed: !bIsInlined || BulkMeta.GetOffset() == INDEX_NONE || IgnoreInlineDataReloadEnsures.bEnabled  [File:D:/build/++UE5/Sync/Engine/Source/Runtime/CoreUObject/Private/Serialization/BulkDataStreaming.cpp] [Line: 128]
LogOutputDevice: Error: Attempting to stream inline BulkData! This operation is not supported by the IoDispatcher and so will eventually stop working. The calling code should be fixed to retain the inline data in memory and re-use it rather than discard it and then try to reload from disk!
LogOutputDevice: Error: Stack:
LogOutputDevice: Error: [Callstack] 0x00000071E35F0C9C libUnreal.so(0x000000000D60DC9C)!UE::BulkData::Private::EnsureCanStreamBulkData(UE::BulkData::Private::FBulkMetaData const&)  []
LogOutputDevice: Error: [Callstack] 0x00000071E35F25E0 libUnreal.so(0x000000000D60F5E0)!UE::BulkData::Private::OpenReadBulkData(UE::BulkData::Private::FBulkMetaData const&, UE::BulkData::Private::FBulkDataChunkId const&, long long, long long, EAsyncIOPriorityAndFlags, TFunction<void (FArchive&)>&&)  []
LogOutputDevice: Error: [Callstack] 0x00000071E35CC4B0 libUnreal.so(0x000000000D5E94B0)!FBulkData::TryLoadDataIntoMemory(FIoBuffer)  []
LogOutputDevice: Error: [Callstack] 0x00000071E35CC940 libUnreal.so(0x000000000D5E9940)!FBulkData::MakeSureBulkDataIsLoaded()  []
LogOutputDevice: Error: [Callstack] 0x00000071E35CC7AC libUnreal.so(0x000000000D5E97AC)!FBulkData::Lock(unsigned int)  []
LogOutputDevice: Error: [Callstack] 0x00000071E057CA4C libUnreal.so(0x000000000A599A4C)!FGoogleARCoreXRTrackingSystem::OnAddRuntimeCandidateImage(UARSessionConfig*, UTexture2D*, FString, float)  []
LogOutputDevice: Error: [Callstack] 0x00000071E53D5598 libUnreal.so(0x000000000F3F2598)!FARSupportInterface::AddRuntimeCandidateImage(UARSessionConfig*, UTexture2D*, FString, float)  []
LogOutputDevice: Error: [Callstack] 0x00000071E53C99EC libUnreal.so(0x000000000F3E69EC)!UARBlueprintLibrary::AddRuntimeCandidateImage(UARSessionConfig*, UTexture2D*, FString, float)  []
LogOutputDevice: Error: [Callstack] 0x00000071E53C97F4 libUnreal.so(0x000000000F3E67F4)!UARBlueprintLibrary::execAddRuntimeCandidateImage(UObject*, FFrame&, void*)  []
LogOutputDevice: Error: [Callstack] 0x00000071E38B7D34 libUnreal.so(0x000000000D8D4D34)!UObject::execCallMathFunction(UObject*, FFrame&, void*)  []
LogOutputDevice: Error: [Callstack] 0x00000071E38C5DA0 libUnreal.so(0x000000000D8E2DA0)!UObject::execLetObj(UObject*, FFrame&, void*)  []
LogOutputDevice: Error: [Callstack] 0x00000071E38BA4F0 libUnreal.so(0x000000000D8D74F0)!ProcessLocalScriptFunction(UObject*, FFrame&, void*)  []
LogOutputDevice: Error: [Callstack] 0x00000071E38BA080 libUnreal.so(0x000000000D8D7080)!UObject::ProcessInternal(UObject*, FFrame&, void*)  []
LogOutputDevice: Error: [Callstack] 0x00000071E362CC08 libUnreal.so(0x000000000D649C08)!UFunction::Invoke(UObject*, FFrame&, void*)  []
LogOutputDevice: Error: [Callstack] 0x00000071E38BFC58 libUnreal.so(0x000000000D8DCC58)!UObject::ProcessEvent(UFunction*, void*)  []
LogOutputDevice: Error: [Callstack] 0x00000071E75281B4 libUnreal.so(0x00000000115451B4)!AActor::ProcessEvent(UFunction*, void*)  []
LogOutputDevice: Error: [Callstack] 0x00000071E7FE15D0 libUnreal.so(0x0000000011FFE5D0)!FLatentActionManager::TickLatentActionForObject(float, TMultiMap<int, FPendingLatentAction*, FDefaultSetAllocator, TDefaultMapHashableKeyFuncs<int, FPendingLatentAction*, true> >&, UObject*)  []
LogOutputDevice: Error: [Callstack] 0x00000071E7FE0EFC libUnreal.so(0x0000000011FFDEFC)!FLatentActionManager::ProcessLatentActions(UObject*, float)  []
LogOutputDevice: Error: [Callstack] 0x00000071E75294AC libUnreal.so(0x00000000115464AC)!AActor::Tick(float)  []
LogOutputDevice: Error: [Callstack] 0x00000071E7524A20 libUnreal.so(0x0000000011541A20)!FActorTickFunction::ExecuteTick(float, ELevelTick, ENamedThreads::Type, TRefCountPtr<FGraphEvent> const&)  []
LogOutputDevice: Error: [Callstack] 0x00000071E8AF3D14 libUnreal.so(0x0000000012B10D14)!FTickFunctionTask::DoTask(ENamedThreads::Type, TRefCountPtr<FGraphEvent> const&)  []
LogOutputDevice: Error: [Callstack] 0x00000071E8AF36F0 libUnreal.so(0x0000000012B106F0)!TGraphTask<FTickFunctionTask>::ExecuteTask(TArray<FBaseGraphTask*, TSizedDefaultAllocator<32> >&, ENamedThreads::Type, bool)  []
LogOutputDevice: Error: [Callstack] 0x00000071E2D36D4C libUnreal.so(0x000000000CD53D4C)!FNamedTaskThread::ProcessTasksNamedThread(int, bool)  []
LogOutputDevice: Error: [Callstack] 0x00000071E2D35008 libUnreal.so(0x000000000CD52008)!FNamedTaskThread::ProcessTasksUntilQuit(int)  []
LogOutputDevice: Error: [Callstack] 0x00000071E2D340C8 libUnreal.so(0x000000000CD510C8)!FTaskGraphCompatibilityImplementation::ProcessThreadUntilRequestReturn(ENamedThreads::Type)  []
LogOutputDevice: Error: [Callstack] 0x00000071E2D34660 libUnreal.so(0x000000000CD51660)!FTaskGraphCompatibilityImplementation::WaitUntilTasksComplete(TArray<TRefCountPtr<FGraphEvent>, TSizedInlineAllocator<4u, 32, TSizedDefaultAllocator<32> > > const&, ENamedThreads::Type)  []
LogOutputDevice: Error: [Callstack] 0x00000071E8AEBE08 libUnreal.so(0x0000000012B08E08)!FTickTaskSequencer::ReleaseTickGroup(ETickingGroup, bool)  []
LogOutputDevice: Error: [Callstack] 0x00000071E8AE6D4C libUnreal.so(0x0000000012B03D4C)!FTickTaskManager::RunTickGroup(ETickingGroup, bool)  []
LogOutputDevice: Error: [Callstack] 0x00000071E8043340 libUnreal.so(0x0000000012060340)!UWorld::Tick(ELevelTick, float)  []
LogOutputDevice: Error: [Callstack] 0x00000071E7D65F0C libUnreal.so(0x0000000011D82F0C)!UGameEngine::Tick(float, bool)  []
LogOutputDevice: Error: [Callstack] 0x00000071E050B510 libUnreal.so(0x000000000A528510)!FEngineLoop::Tick()  []
LogOutputDevice: Error: [Callstack] 0x00000071E0503EFC libUnreal.so(0x000000000A520EFC)!AndroidMain(android_app*)  []
LogOutputDevice: Error: [Callstack] 0x00000071E0512448 libUnreal.so(0x000000000A52F448)!android_main()  []
LogOutputDevice: Error: [Callstack] 0x00000071E0549380 libUnreal.so(0x000000000A566380)![Unknown]()  []
LogOutputDevice: Error: [Callstack] 0x00000072DF839F24 libc.so(0x00000000000E6F24)![Unknown]()  []
LogOutputDevice: Error: [Callstack] 0x00000072DF7D80CC libc.so(0x00000000000850CC)![Unknown]()  []

BP_ARPawn is the ARPawn Blueprint that I use. I init the AR Session here.

SilvRO · May 11, 2023, 3:18pm

If anyone is interested:

The blueprint

The data struct