MediaIOFramework regression in 5.6

Development Cinematics & Media Virtual Production
, ,
1

The changes in MediaIOFramework to avoid fences when capturing textures breaks our workflow. We are seeing latency between the captured texture and the FMediaCaptureUserData.

If I apply these changes everything is back to normal and I get the same results as in 5.5:

`diff --git a/Engine/Plugins/Media/MediaIOFramework/Source/MediaIOCore/Private/MediaCapture.cpp b/Engine/Plugins/Media/MediaIOFramework/Source/MediaIOCore/Private/MediaCapture.cpp
index db5dc023bab9..0ec022a9aeca 100644
— a/Engine/Plugins/Media/MediaIOFramework/Source/MediaIOCore/Private/MediaCapture.cpp
+++ b/Engine/Plugins/Media/MediaIOFramework/Source/MediaIOCore/Private/MediaCapture.cpp
[Content removed]7 @@ void UMediaCapture::InitializeSyncHandlers_RenderThread()
for (int32 Index = 0; Index < NumberOfCaptureFrame; ++Index)
{
TSharedPtr SyncData = MakeShared();

  •   if (!ShouldCaptureRHIResource())

  •   if (1) //!ShouldCaptureRHIResource())
  {
      SyncData->RHIFence = RHICreateGPUFence(*FString::Printf(TEXT("MediaCaptureSync_%02d"), Index));
  }

diff --git a/Engine/Plugins/Media/MediaIOFramework/Source/MediaIOCore/Private/MediaCaptureHelper.cpp b/Engine/Plugins/Media/MediaIOFramework/Source/MediaIOCore/Private/MediaCaptureHelper.cpp
index 4e1d134ff0c6..fb215457a7b3 100644
— a/Engine/Plugins/Media/MediaIOFramework/Source/MediaIOCore/Private/MediaCaptureHelper.cpp
+++ b/Engine/Plugins/Media/MediaIOFramework/Source/MediaIOCore/Private/MediaCaptureHelper.cpp
[Content removed]7 @@ void FMediaCaptureHelper::ExecuteSyncPointPass(FRHICommandListImmediate& RHICmdL
{
if (CapturingFrame && CapturingFrame->bMediaCaptureActive)
{

  •   if (MediaCapture->ShouldCaptureRHIResource())

  •   if (0) //MediaCapture->ShouldCaptureRHIResource())
  {
      MediaCapture->UnlockDMATexture_RenderThread(CapturingFrame->GetTextureResource());`
2

Hi!

Would you mind clarifying what you mean by “Latency between the captured texture and the FMediaCaptureUserData”?

Looking at the code, it seems like this code should only be executing if you explicitly enabled GPUDirect. Was that the case?

Furthermore, what Media Output are you using? Aja, Blackmagic or something else?

Cheers,

Jérémie

3

Hi,

I mean the captured texture and the user data, as seen from our external application, don’t belong to the same frame.

We are using our own plugin to transfer the renders (and the corresponding camera information) to another application via shared memory + GPU texture transfer (kind of USharedMemoryMediaCapture).

Our plugin uses:

bool ShouldCaptureRHIResource() const override { return true; } bool SupportsAnyThreadCapture() const override { return true; }Thanks,

Jorge

4

Hi,

I forgot to say this is a blocking issue for us in order to release a UE5.6-based product. We can’t apply that workaround because we don’t distribute UE and just use a standard install.

Regards,

Jorge

5

Hi Jorge,

At this stage, we’ve passed the cutoff for submitting fixes to 5.6.1. However, I’ll go ahead and submit a change that allows reverting to the previous workflow. Unfortunately, that update will only be available to customers in the 5.7 Preview 1 release.

Let me know if you have any questions in the meantime.

Jérémie