Best way to debug deleted proxies.

Development Programming & Scripting
, , ,
1

Hey!

I have a bit of a newbie question. I’m dealing with a weird issue in my project (5.4) where I occasionally get a Chaos crash in seemingly random places—usually a segmentation fault or access violation when reading ShapesData, Proxy, or something similar.

This makes me suspect that something is deleting the proxy (or whatever data format is in use at that moment), but since the crash happens on a different thread, the call stack isn’t helpful. Randomly adding logs to track deletions also seems like a nightmare.

What’s the best way to figure out what deleted an object that I had a proxy for? Could it be GC, a random Blueprint, or something related to streaming? The issue is so rare that I can’t just flood my code with logs and wait for it to happen.

(I could paste in a specific crash callstack, but it would be just a read error on a random part of the PBDResolver, or one of the header files).

Any good suggestions? :frowning:

2

Greetings @Bobdev_do!

It would depend on the nature of the error, and if its happening a particular project, or simply anyfresh project you start. If that is the case, I would suggest a full UE refresh:

  • First, uninstall the engine, launcher, asset packs, etc
  • Next, open the registry directory HKEY_CURRENT_USER\Software\Epic Games\Unreal Engine\Builds, and delete any build entries
  • Finally, delete the folders %userprofile%\AppData\Local\UnrealEngine and %userprofile%\AppData\Local\UnrealHeaderTool
  • After that, reinstall UE from scratch, all settings should be at default values

On the other hand, if the issue is isolated to a single project, then please include the full error logs, either from the crash window, or from AppData\Local\CrashReportClient\Saved\Logs

3

Thank you very much for your reply!

It is a project specific issue, but as I said the crash log or the callstack is different for every crash, no point to paste one here.

This question is more about a generic approach, how would you debug a crash that is a few frames behind of the actual cause, and on a different thread.

4

Hi,

If you can share the callstack, that would help us to know which line of code is trying to access the invalid data, and based on that we might be able to provide some suggestions about what scenario might be causing it.

You are probably getting the crash in different threads each time, because the system crashing is the one working with the physics thread state of the simulation. In Chaos, we have two states of the simulation. One is the Game thread state (this is the one you have access trough the existing API to interact with physics from Actor/Components), and we also have the physics thread state. That last one is the one used to advance the simulation using multiple threads, not just one (and not the game thread).

Proxies are part of the system used to sync back and forth between the two. I recently explained a bit how this works in another post Is Async Physics Tick really async? - #2 by sergio.gardeazabal .

That said,

If the code trying to access the (now invalid) proxy is doing it trough a particle handle object, you might be able to inspect in the debugger the DebugName property (if you are able to reproduce the crash with the debugger connected).

In Debug/Development builds all Chaos particles (Physics Bodies) should have a debug name property. Which is a string composed of the ActorName+PrimitiveComponentName+BoneID.

2 Likes
5

Thank you very much, super useful info, both the reply and the async one :)!

What makes it really challenging that it is a fairly rare crash :frowning: around 5% repro rate, but that happens, and mostly on server side, so still pretty bad, and I cant just repro it on demond to attach a debugger. So my best bet would be some kind of a logger I assume.

When it happens, most of the time its during the sync process. Here is one of the callstacks, in this case a ShapeInstanceProxy (but I have almost every type of proxy, this and the GeometryParticle version is the most common crash).

SIGSEGV / SEGV_MAPPERR - fatal error

Chaos::FShapeInstanceProxy::SyncRemoteData (ShapeInstance.h:407) <- 
Chaos::TGeometryParticle<T>::SyncRemoteData (ParticleHandle.h:3007)
Chaos::FPBDRigidsSolver::PuishPhysicsState::'2'::<T>::Operator()
UE::Core::Private::Function::TFunctionRefBase><T>::operator() (Function.h:555)
ParallelForImpl::CallBody (ParallelFor.h:80)
ParallelForImpl::ParallelForInternal<T> - FParallelExecutor::operator() (ParallelFor.h:353)
LowLevelTasks::FTask::Init (Task.h:511)
.
.
.
LowLevelTasks::TTaskDelegate<T>::TTaskDelegateImpl<T>:Call (TaskDelegate.h:162)
LowLevelTasks::TTaskDelegate<T>::TTaskDelegateImpl<T>:CallAndMove (TaskDelegate.h:171)
LowLevelTasks::TTaskDelegate<T>::CallAndMove (TaskDelegate.h:308)
LowLevelTasks::FTask::ExecuteTask (Task.h:618)
6

You are welcome :slight_smile:

I see, the next thing that comes to mind is try to get a minidump from the crash to see if there any additional info there (basically to see what was invalid, the particle or the shape data. But it might not have the data we need).

The minidump is usually located at ProjectName/Saved/Crashes

Is your project welding/unwelding body instances (or atacching/detaching components) in runtime?

That is one of the code paths that will alter the shapes array, without necessarily involving a component being destroyed.