Announcement

Collapse
No announcement yet.

Copy backbuffer to texture

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Copy backbuffer to texture

    Hello! I'm implementing some features and i need to copy backbuffer to rendertarget (and next resize/blur it, but it doesn't matter, i just need a copy of backbuffer). Of course I'm looking for a solution to do that with GPU/GPU Ram only (without extra coping texture data to RAM and the usage of CPU or blocking game thread). I've found event that is fired from rendering pipeline and succesfully subscribed that:

    Code:
    FDelegateHandle OnBackBufferReadyToPresent = FSlateApplication::Get().GetRenderer()->OnBackBufferReadyToPresent().AddUObject(this, &UCachedFrameBufferComponent::OnBackBufferReady_RenderThread);
    It seems that I'm able to get the access to backbuffer here and copy it to created render target with copy texture command, but that doesn't work (back buffer contains data that i need, but i cannot copy that to my rendertarget):

    Code:
    void UCachedFrameBufferComponent::OnBackBufferReady_RenderThread(SWindow& SlateWindow, const FTexture2DRHIRef& BackBuffer)
    {
    	ensure(IsInRenderingThread());
    
    	FRHICommandListImmediate& RHICmdList = GRHICommandList.GetImmediateCommandList();
    	FRHITexture2D* CachedTexture = CachedFrameBuffer->Resource->TextureRHI->GetTexture2D();
    
    	FRHICopyTextureInfo CopyInfo;
    	RHICmdList.CopyTexture(CachedTexture, BackBuffer, CopyInfo);
    }
    CachedFrameBuffer is blank. If it makes sense I create CachedFrameBuffer from editor and tried different resolutions (including back buffer resolution), formats and etc. Moreover, I tried to create it manually and this doesn't affect too.
    Code:
    UTextureRenderTarget2D* CachedFrameBuffer = UKismetRenderingLibrary::CreateRenderTarget2D(GetOwner()->GetWorld(), ViewportSize.X * ScreenPercentage, ViewportSize.Y * ScreenPercentage, ETextureRenderTargetFormat::RTF_RGBA8);
    After investigation of engine's sources I find more progressive way to do that:

    Code:
    void UCachedFrameBufferComponent::CopyBackBuffer(const FTexture2DRHIRef& BackBuffer, const FTexture2DRHIRef& CachedFrame)
    {
    	IRendererModule* RendererModule = &FModuleManager::GetModuleChecked<IRendererModule>("Renderer");
    	FRHICommandListImmediate& RHICmdList = FRHICommandListExecutor::GetImmediateCommandList();
    
    	if (BackBuffer->GetFormat() == CachedFrame->GetFormat() &&
    		BackBuffer->GetSizeXY() == CachedFrame->GetSizeXY())
    	{
    		RHICmdList.CopyToResolveTarget(BackBuffer, CachedFrame, FResolveParams{});
    	}
    	else // Texture format mismatch, use a shader to do the copy.
    	{
    		// #todo-renderpasses there's no explicit resolve here? Do we need one?
    		FRHIRenderPassInfo RPInfo(CachedFrame, ERenderTargetActions::Load_Store);
    		RHICmdList.BeginRenderPass(RPInfo, TEXT("CopyBackbuffer"));
    		{
    			RHICmdList.SetViewport(0, 0, 0.0f, CachedFrame->GetSizeX(), CachedFrame->GetSizeY(), 1.0f);
    
    			FGraphicsPipelineStateInitializer GraphicsPSOInit;
    			RHICmdList.ApplyCachedRenderTargets(GraphicsPSOInit);
    			GraphicsPSOInit.BlendState = TStaticBlendState<>::GetRHI();
    			GraphicsPSOInit.RasterizerState = TStaticRasterizerState<>::GetRHI();
    			GraphicsPSOInit.DepthStencilState = TStaticDepthStencilState<false, CF_Always>::GetRHI();
    
    			TShaderMap<FGlobalShaderType>* ShaderMap = GetGlobalShaderMap(GMaxRHIFeatureLevel);
    			TShaderMapRef<FScreenVS> VertexShader(ShaderMap);
    			TShaderMapRef<FScreenPS> PixelShader(ShaderMap);
    
    			GraphicsPSOInit.BoundShaderState.VertexDeclarationRHI = GFilterVertexDeclaration.VertexDeclarationRHI;
    			GraphicsPSOInit.BoundShaderState.VertexShaderRHI = GETSAFERHISHADER_VERTEX(*VertexShader);
    			GraphicsPSOInit.BoundShaderState.PixelShaderRHI = GETSAFERHISHADER_PIXEL(*PixelShader);
    			GraphicsPSOInit.PrimitiveType = PT_TriangleList;
    
    			SetGraphicsPipelineState(RHICmdList, GraphicsPSOInit);
    
    			if (CachedFrame->GetSizeX() != BackBuffer->GetSizeX() || CachedFrame->GetSizeY() != BackBuffer->GetSizeY())
    			{
    				PixelShader->SetParameters(RHICmdList, TStaticSamplerState<SF_Bilinear>::GetRHI(), BackBuffer);
    			}
    			else
    			{
    				PixelShader->SetParameters(RHICmdList, TStaticSamplerState<SF_Point>::GetRHI(), BackBuffer);
    			}
    
    			RendererModule->DrawRectangle(
    				RHICmdList,
    				0, 0,									// Dest X, Y
    				CachedFrame->GetSizeX(),			// Dest Width
    				CachedFrame->GetSizeY(),			// Dest Height
    				0, 0,									// Source U, V
    				1, 1,									// Source USize, VSize
    				CachedFrame->GetSizeXY(),		// Target buffer size
    				FIntPoint(1, 1),						// Source texture size
    				*VertexShader,
    				EDRF_Default);
    		}
    		RHICmdList.EndRenderPass();
    	}
    }
    The approach is quite obvious: bind render target and render back buffer texture to it. But it doesn't work, my render target still is blank.

    Maybe I'm missing something during operating with RHI texture objects instead of Unreal's UTextureRenderTarget2D*?
    Code:
    FRHITexture2D* CachedTexture = CachedFrameBuffer->Resource->TextureRHI->GetTexture2D();
    Also I tried to wrap RHI commands with ENQUEUE_RENDER_COMMAND macros,(btw should i do this for code that is run in rendering thread in BackBufferReady callback? I think not, anyway that doesn't help, render target still is blank).

    So, please help me or share the info where I should dig in engine to achieve that I need.

    Thank you!

    #2
    Also, If it makes sense I use the latest version of Engine: 4.24.1

    Comment


      #3
      Just carefully redo that in test project, and that works. Seems there was a silly misprint/mistake, probably in render texture initialization. The correct code is next:

      Code:
      void ACacheFrameBufferActor::BeginPlay()
      {
          Super::BeginPlay();
      
          OnBackBufferReadyToPresent = FSlateApplication::Get().GetRenderer()->OnBackBufferReadyToPresent().AddUObject(this, &ACacheFrameBufferActor::OnBackBufferReady_RenderThread);
          bIsCalculatedCaptureRect = CalculateCaptureRect(CaptureRect);
      
          CaptureFrameTexture = UKismetRenderingLibrary::CreateRenderTarget2D(GetWorld(), 128, 128);
          MaterialInstanceDynamic->SetTextureParameterValue(FName("Texture"), CaptureFrameTexture);
      }
      
      bool ACacheFrameBufferActor::CalculateCaptureRect(FIntRect& CaptureRect) const
      {
      
      #if WITH_EDITOR
      
          TSharedPtr<FSceneViewport> SceneViewport;
      
          if (GIsEditor)
          {
              for (const FWorldContext& Context : GEngine->GetWorldContexts())
              {
                  if (Context.WorldType == EWorldType::PIE)
                  {
                      FSlatePlayInEditorInfo* SlatePlayInEditorSession = GEditor->SlatePlayInEditorMap.Find(Context.ContextHandle);
                      if (SlatePlayInEditorSession)
                      {
                          if (SlatePlayInEditorSession->DestinationSlateViewport.IsValid())
                          {
                              TSharedPtr<IAssetViewport> DestinationLevelViewport = SlatePlayInEditorSession->DestinationSlateViewport.Pin();
      
                              SceneViewport = DestinationLevelViewport->GetSharedActiveViewport();
                          }
                          else if (SlatePlayInEditorSession->SlatePlayInEditorWindowViewport.IsValid())
                          {
                              SceneViewport = SlatePlayInEditorSession->SlatePlayInEditorWindowViewport;
                          }
                      }
                  }
              }
          }
      
          if (SceneViewport == nullptr || !SceneViewport.IsValid())
          {
              return false;
          }
      
          CaptureRect = FIntRect(0, 0, SceneViewport->GetSize().X, SceneViewport->GetSize().Y);
          FIntPoint WindowSize(0, 0);
      
          // Set up the capture rectangle
          TSharedPtr<SViewport> ViewportWidget = SceneViewport->GetViewportWidget().Pin();
          if (ViewportWidget.IsValid())
          {
              TSharedPtr<SWindow> Window = FSlateApplication::Get().FindWidgetWindow(ViewportWidget.ToSharedRef());
              if (Window.IsValid())
              {
                  //TargetWindowPtr = Window.Get();
                  FGeometry InnerWindowGeometry = Window->GetWindowGeometryInWindow();
      
                  // Find the widget path relative to the window
                  FArrangedChildren JustWindow(EVisibility::Visible);
                  JustWindow.AddWidget(FArrangedWidget(Window.ToSharedRef(), InnerWindowGeometry));
      
                  FWidgetPath WidgetPath(Window.ToSharedRef(), JustWindow);
                  if (WidgetPath.ExtendPathTo(FWidgetMatcher(ViewportWidget.ToSharedRef()), EVisibility::Visible))
                  {
                      FArrangedWidget ArrangedWidget = WidgetPath.FindArrangedWidget(ViewportWidget.ToSharedRef()).Get(FArrangedWidget::GetNullWidget());
      
                      FVector2D Position = ArrangedWidget.Geometry.GetAbsolutePosition();
                      FVector2D Size = ArrangedWidget.Geometry.GetAbsoluteSize();
      
                      CaptureRect = FIntRect(
                          Position.X,
                          Position.Y,
                          Position.X + Size.X,
                          Position.Y + Size.Y);
      
                      FVector2D AbsoluteSize = InnerWindowGeometry.GetAbsoluteSize();
                      WindowSize = FIntPoint(AbsoluteSize.X, AbsoluteSize.Y);
      
                  }
              }
          }
      
          return true;
      
      #endif
      
          return false;
      }
      
      void ACacheFrameBufferActor::OnBackBufferReady_RenderThread(SWindow& SlateWindow, const FTexture2DRHIRef& BackBuffer)
      {
          ensure(IsInRenderingThread());
      
          if (CaptureFrameTexture != nullptr)
          {
              FRHICommandListImmediate& RHICmdList = GRHICommandList.GetImmediateCommandList();
              FRHITexture2D* CachedTexture = CaptureFrameTexture->Resource->TextureRHI->GetTexture2D();
      
              FBox2D Region = FBox2D(FVector2D::ZeroVector, FVector2D::UnitVector);
      
              if (bIsCalculatedCaptureRect)
              {
                  float U = float(CaptureRect.Min.X) / float(BackBuffer->GetSizeX());
                  float V = float(CaptureRect.Min.Y) / float(BackBuffer->GetSizeY());
                  float SizeU = float(CaptureRect.Max.X) / float(BackBuffer->GetSizeX());
                  float SizeV = float(CaptureRect.Max.Y) / float(BackBuffer->GetSizeY());
      
                  Region = FBox2D(FVector2D(U,V), FVector2D(SizeU, SizeV));
              }
      
              CopyBackBuffer(BackBuffer, CachedTexture, Region);
          }
      }
      
      void ACacheFrameBufferActor::CopyBackBuffer(const FTexture2DRHIRef& BackBuffer, const FTexture2DRHIRef& CachedFrame, const FBox2D& BackBufferRegion)
      {
          IRendererModule* RendererModule = &FModuleManager::GetModuleChecked<IRendererModule>("Renderer");
          FRHICommandListImmediate& RHICmdList = FRHICommandListExecutor::GetImmediateCommandList();
      
          if (BackBuffer->GetFormat() == CachedFrame->GetFormat() &&
              BackBuffer->GetSizeXY() == CachedFrame->GetSizeXY())
          {
              RHICmdList.CopyToResolveTarget(BackBuffer, CachedFrame, FResolveParams{});
          }
          else // Texture format mismatch, use a shader to do the copy.
          {
              // #todo-renderpasses there's no explicit resolve here? Do we need one?
              FRHIRenderPassInfo RPInfo(CachedFrame, ERenderTargetActions::Load_Store);
              RHICmdList.BeginRenderPass(RPInfo, TEXT("CopyBackbuffer"));
              {
                  RHICmdList.SetViewport(0, 0, 0.0f, CachedFrame->GetSizeX(), CachedFrame->GetSizeY(), 1.0f);
      
                  FGraphicsPipelineStateInitializer GraphicsPSOInit;
                  RHICmdList.ApplyCachedRenderTargets(GraphicsPSOInit);
                  GraphicsPSOInit.BlendState = TStaticBlendState<>::GetRHI();
                  GraphicsPSOInit.RasterizerState = TStaticRasterizerState<>::GetRHI();
                  GraphicsPSOInit.DepthStencilState = TStaticDepthStencilState<false, CF_Always>::GetRHI();
      
                  TShaderMap<FGlobalShaderType>* ShaderMap = GetGlobalShaderMap(GMaxRHIFeatureLevel);
                  TShaderMapRef<FScreenVS> VertexShader(ShaderMap);
                  TShaderMapRef<FScreenPS> PixelShader(ShaderMap);
      
                  GraphicsPSOInit.BoundShaderState.VertexDeclarationRHI = GFilterVertexDeclaration.VertexDeclarationRHI;
                  GraphicsPSOInit.BoundShaderState.VertexShaderRHI = GETSAFERHISHADER_VERTEX(*VertexShader);
                  GraphicsPSOInit.BoundShaderState.PixelShaderRHI = GETSAFERHISHADER_PIXEL(*PixelShader);
                  GraphicsPSOInit.PrimitiveType = PT_TriangleList;
      
                  SetGraphicsPipelineState(RHICmdList, GraphicsPSOInit);
      
                  if (CachedFrame->GetSizeX() != BackBuffer->GetSizeX() || CachedFrame->GetSizeY() != BackBuffer->GetSizeY())
                  {
                      PixelShader->SetParameters(RHICmdList, TStaticSamplerState<SF_Bilinear>::GetRHI(), BackBuffer);
                  }
                  else
                  {
                      PixelShader->SetParameters(RHICmdList, TStaticSamplerState<SF_Point>::GetRHI(), BackBuffer);
                  }
      
                  FVector2D Size = BackBufferRegion.GetSize();
      
                  RendererModule->DrawRectangle(
                      RHICmdList,
                      0, 0,                                                // Dest X, Y
                      CachedFrame->GetSizeX(),                            // Dest Width
                      CachedFrame->GetSizeY(),                            // Dest Height
                      BackBufferRegion.Min.X, BackBufferRegion.Min.Y,
                      Size.X, Size.Y,                                        // Source USize, VSize
                      CachedFrame->GetSizeXY(),                            // Target buffer size
                      FIntPoint(1, 1),                                    // Source texture size
                      *VertexShader,
                      EDRF_Default);
              }
              RHICmdList.EndRenderPass();
          }
      }

      Comment

      Working...
      X