Rendering a Secondary View from a Different ViewOrigin Using SceneViewExtension Without Affecting Main View?

Development Rendering
, , , , , , , , , , , , , , ,
1

I’m working on a project where I need to render the scene from a different ViewOrigin without affecting the main camera view. My goal is to set up a secondary view that renders the scene from an alternate location and orientation (Without SceneCapture2D), and then access its SceneTextures (like SceneColor and SceneDepth) directly within the rendering pipeline.

What I’m Trying to Achieve:

  • Render a Secondary View:
    • Create a new view that renders the scene from a different ViewOrigin and ViewRotation.
    • Ensure that this secondary view does not interfere with or alter the main camera’s rendering.
  • Access SceneTextures of the Secondary View:
    • Retrieve the SceneColor and SceneDepth textures from the secondary view.
    • Use these textures in custom render passes or shaders.

What I’ve Done So Far:

  • SceneViewExtension:
    • Implemented an FSceneViewExtension class to inject custom rendering logic.
    • Overridden necessary methods like PrePostProcessPass_RenderThread to access and manipulate SceneTextures.
  • Custom Render Passes and Shaders:
    • Set up custom render passes and global shaders that are working as intended.
    • Able to manipulate the main view’s SceneTextures successfully.

The Challenge:

I’m stumped on how to properly render the secondary view from the different ViewOrigin and access its SceneTextures within the rendering pipeline. Specifically:

  1. Creating the Secondary View:
  • How to set up and render a new FSceneView or FViewInfo from the alternate location within the existing rendering flow.
  • Ensuring that rendering this view does not affect the main camera’s view.

What I’ve Tried:

  • Attempted to Create a New View in PrePostProcessPass_RenderThread, SetupView, SetupViewFamily:
    • Created a new FSceneViewFamily and FViewInfo for the secondary view.
    • Faced compilation errors and runtime issues, likely due to incorrect setup or misuse of the rendering API.
  • Considered Rendering to a UTextureRenderTarget2D:
    • Thought about rendering the secondary view to a separate render target.
    • Uncertain how to integrate this render target back into the render graph to access its SceneTextures.

Specific Questions:

  1. How can I correctly set up and render a secondary view from a different ViewOrigin within the rendering pipeline using SceneViewExtension?
  2. Are there any examples or recommended practices for rendering additional views and accessing their textures without affecting the main view?

Additional Information:

  • Engine Version: Unreal Engine 5.4.
  • Constraints:
    • I’m not using USceneCaptureComponent2D because I prefer to handle this within the rendering pipeline directly for performance and integration reasons.
    • I already have custom render passes and shaders working; I just need help with rendering the secondary view and accessing its textures.

Any guidance, examples, or suggestions would be greatly appreciated!

Thank you for your time and assistance.

2

Any ideas?

3

bump, I have not figured this out yet.

4

I’m a mashup of rusty, outdated… and not entirely confident in my knowledge here. But, have you taken a peek at documentation on the RDG (render dependency graph)? That and/or creating a SceneViewExtension…

I bring these up, because I stumbled across this thread while researching a solution for something I’m working on currently, and thought perhaps some keywords might help you on your hunt.

Either way, if you figure it out, please circle back and detail your solution! While personally I’m not going after the same thing as you are, I can only imagine at some point I might, and certainly others may find the wisdom helpful.

Good luck!