Lost multicast delegate binding

Development Programming & Scripting
, ,
1

Hey!

We have an intermittent issues that might be related to dynamic multicast delegates. It’s quite rare so don’t have to much details for now.

Our setup is a “model” in pure c++ with a APIGear “interface” to our views (a UUserWidget or AActor). Basically then a plugin that configures a DECLARE_DYNAMIC_MULTICAST_DELEGATE and OnChange.Broadcast(InParameter). On the view something like

IViewModelInterface::Execute__GetSignals(GetGameInstance()->GetSubsystem<ViewModelAlias>().GetObject())->OnChanged.AddDynamic(this, &UClass::OnChange);

When we have the issue, we see logs from 2/3 views (in the corresponding OnChange functions) - and assumed to be after a SuspendToRam. No active actions are taken on the views/viewmodels on STR though - stop rendering and set gamethread in a while(STR) to stop execution.

It has worked even in the failing cases, since the displayed state is not the default (eg. no binding errors).

One of the resent changes are an Engine update, from perforce CL 44570631 -> CL 50934014 (Private-HMI-4.27). Could be one suspect, but don’t see any obvious change.

Any ideas on how to proceed or seen similar issues? Have added more logs, but kind of lost for now.

Thanks!

//Jonas

[Attachment Removed]

2

Hi,

Are you able to tell if it always the same object that loses it’s binding (i.e. a UserWidget or an Actor), or does it vary? Where exactly are you setting up the binding? One common issue with UMG widgets specifically is that they may call Construct/Destruct multiple times (as they build/destroy their underlying Slate widget) so sometimes bindings can get a bit mixed up. Ideally, any one-off logic like binding delegates can happen in OnInitialized.

The other thought would be to add some extra tracking as you bind things, perhaps log out the memory address of the UObject that you’re using for the binding and comparing that to the list of bindings present (and the objects that expect to be bound) when one fails so you can narrow down whether the binding has been lost or the object has been reinstanced (and you’re now bound to an old instance).

Is there anything unusual in the log when the failure happens? Something like a memory stomp could also break the invocation list, though I’d expect that to fail less gracefully.

Best,

Cody

[Attachment Removed]

3

Thanks!

We have some occurrences that can be related, but more on the level that some hmi state should have been updated, but have not. But no good logs to see the complete chain. As far as we’ve seen the issue is healed, so it’s more that like one event (or some) is missed.

We do all of the bindings in NativeConstruct(), and get the initial value. And unbinding in NativeDestruct(), but the widgets are never deleted, so should probably never be called on target.

We are adding logs and trying to reproduce.

//Jonas

[Attachment Removed]

4

Sounds good, let us know if you’re able to track down more information. I do wonder if setting up the bindings on NativeConstruct is a factor, so some logging there might reveal something. We destroy the Slate widget whenever it’s not visible, so perhaps the SuspendToRam is interacting with that in some nondeterministic way and sometimes leading to the binding being lost.

[Attachment Removed]

5

Think we have 2 different issues, on is then after STR (waiting with the other for now not to confuse). We’ve added logs and check on “OnModeUpdated.IsBound()” before triggering broadcast. When we have the issue, there are no bindings (three are listning on that). This issue is not recoverable. The suspend looks like this

FlushRenderingCommands();
FSuspendRenderingThread Suspend(true);
 
while (GetApplicationLockout()) {
        FPlatformProcess::Sleep(0.1f);
}

So when we get back, the while loop will exit, and gamethread will continue. So we are restarting the rendering thread, not sure if that can have any impact. One thing that has changed recently, is that we are triggering the the delegate almost directly when coming out of STR. Also this is just seen (so far) on customer builds - our application is shipping also in other builds - but this might have some timing effects. We have logs also in the NativeDestruct, and don’t see any of those. Also the hmi components are visible.

It’s a critical issue that need to be fixed asap, so any ideas are appreciate

Thanks!

//Jonas

[Attachment Removed]

6

One more difference with the latest changes are that it’s not the gamethread-context that the delegate broadcast is made in (as it is usually). Can that mess up? Don’t think gamethread should trigger the same delegate at the same time, but might also be bad, or are they thread safe? Could GC kick in and clean up maybe? Thinking since we get not bound.

//Jonas

[Attachment Removed]

7

Hi,

That could certainly be the cause, as multicast delegates wont be thread safe unless specifically declared to be. If the invocation list is getting corrupted then that explains what you’re seeing. We do support thread-safe multicast delegates now (DECLARE_TS_MULTICAST_DELEGATE) but I don’t think that was present in 4.27, so you may just need to dispatch the actual broadcast to the game thread via AsyncTask. Part of the broadcast does include cleaning up the invocation list, so perhaps that’s where things are breaking.

[Attachment Removed]