[iOS] WKWebView takeSnapshotWithConfiguration and drawViewHierarchyInRect both fail to capture video content for 3D browser texture rendering

Development Programming & Scripting UI
, ,
1

When using drawViewHierarchyInRect:afterScreenUpdates: to capture WKWebView content for rendering to a Metal texture (for 3D browser widget), video elements (such as HTML5 <video> tags or embedded video players) appear as black/blank in the captured image.

Environment:

• Unreal Engine 5.3.2

• iOS 15+ / iOS 16+

• WKWebView with Metal texture rendering

-(void)updateWebViewMetalTexture:(id<MTLTexture>)texture

{

@autoreleasepool {

UIGraphicsBeginImageContextWithOptions(WebView.frame.size, NO, 1\.0f);

\[WebView drawViewHierarchyInRect:WebView.bounds afterScreenUpdates:NO];

UIImage \*image \= UIGraphicsGetImageFromCurrentImageContext();

UIGraphicsEndImageContext();

 

// ... copy image data to Metal texture

\[texture replaceRegion:MTLRegionMake2D(0, 0, width, height)

      mipmapLevel:0

       withBytes:CGBitmapContextGetData(context)

      bytesPerRow:4 \* width];

}

}

I have also tried using WKSnapshotConfiguration with takeSnapshotWithConfiguration, but video content still appears as black.

Questions:

Has anyone successfully captured WKWebView video content to a texture on iOS?

[Attachment Removed]

2

Hi Xinhao,

This is likely a limitation of WKWebView and Hardware Accelerated video playback ending up in a separate layer than the one to which web content gets rendered to resulting in a black rectable in the captured texture. Are you seeing the same behaviour when using SIOSWebBrowserWidget from UE?

Best regards.

[Attachment Removed]