Transfer rendered frame to different application

Development Programming & Scripting C++
, , ,
1

Hello everyone,

I am looking for some pointers to better find my way around the engine source.

I am trying to implement an API provided by a different application that allows me to transfer a rendered frame (or any texture really), so it can be displayed together with some other 3D stuff.
I have already had success by modifying the OpenGLDrv by hooking into RHIBeginDrawingViewport and RHIEndDrawingViewport.
But I would like to make this less hacky and in the editor I would like this to transfer only the scene viewport instead of the entire window.

The required steps look something like this. (Sorry that I can not be more specific, but the API is not officially released)


// all calls assume an active OpenGL context in the case of OpenGL
// Initialize with the native contexts
Init(dxDevice/glContext);

while (1)
{
    // lock frame also gives a delta time which i might want to set as the global delta time (UEngine::UpdateTimeAndHandleMaxTickRate ?)
    frame = LockFrame();

    // do rendering

    // copy the rendered contents into the provided textures
    Blit(renderbuffer, frame.colorTexture);
    Blit(depthRenderbuffer, frame.depthTexture);
    UnlockFrame();
}

//Cleanup resources
Exit();

I am aware of FRHICustomPresent which might be appropriate for this, but I am not sure where I am supposed to assign that to the renderer.
Another idea was to make this a HMD plugin with disabled stereo rendering, since the usecase is very similar.

If someone with more knowledge about the engine source could give me some opinions, that would be great :wink: .

2

Take a look at Spout2

I used it to transfer Unity DX11 Backbuffer to custom DX11 Context in another process in real-time.
The technique relies on resource sharing.

Spout supports OpenGL<>DX sharing however when a certain NVIDIA extension is missing in OpenGL, it will fallback to blitting (slow as hell) as far as I remember.

For OpenGL<>OpenGL it may work fine…

3

Thanks for the link, might come in handy when I need to write a custom app to do stuff like this.

I was actually looking for some info on the unreal engine, as to where I can grab the finished frame and things like that.
The documentation on this seems to be a bit lacking/vague.

Anyway…
I will figure something out to get this to work somehow.

4

What are your speed requirements?
Resolving the backbuffer into a custom byte array for blitting is easy.
For spout you’ll want the native texture id.



FIntPoint imageResolution; 
int32 imageSize; 
std::unique_ptr<FColor]> imageData;

void AMyHUD::DrawHUD() 
{ 
  auto viewport = GEngine−>GameViewport−>Viewport; 

  if (viewport) 
  { 
    auto viewportSize = viewport−>GetSizeXY(); 
  
    if (imageResolution != viewportSize) 
    { 
      imageData.reset(new FColor[viewportSize.X * viewportSize.Y]); 
      imageResolution = viewportSize; 
      imageSize = viewportSize.X * viewportSize.Y * sizeof(FColor); }
    }
    
    viewport−>ReadPixelsPtr(imageData.get() , FReadSurfaceDataFlags());
  }
}
5

What is ‘std::unique_ptr<FColor]> imageData;’? I get :
std namespace has no member unique_ptr