We have a C++ actor (created as a blank actor initially) that acts as a handler of viewport pixels. We have a uint8_t buffer of RGBA values that represents the viewport pixels, and then we do additional processing on said buffer elsewhere.
When using the editor - everything seems to work fine. The buffer is filled with RGBA pixels that represent what is currently being displayed.
However, when packaging the game to a build (we tried both a shipping build and a debug build), the viewport context is valid - but the buffer is filled with zeros - a black frame. Accessing the viewport dimensions seems to work just as expected - resizing the game window changes the height and width, it’s just the RGBA buffer that is black.
Here’s how I acquire the viewport context and try and fill my buffer with pixels:
TArray OutImageData;
UGameViewportClient* GameViewportClient = GEngine ? GEngine->GameViewport : nullptr;
if (!GameViewportClient) { // does NOT reach here
UE_LOG(LogTemp, Error, TEXT(“Could not get GameViewportClient”));
}
GameViewport = GameViewportClient->Viewport;
if (!GameViewport) { // does NOT reach here
UE_LOG(LogTemp, Error, TEXT(“Could not get GameViewport”));
}
FIntPoint ViewportSize = GameViewport->GetSizeXY();
OutImageData.SetNumUninitialized(ViewportSize.X * ViewportSize.Y);
GameViewport->ReadPixels(OutImageData); // OutImageData is a BLACK FRAME, completely zeroed-out. No runtime error
We are aware that there is a UE Pixel Streaming repo, however we couldn’t find how to directly access the streamed pixels without any network communication - we need to access the buffer within a UE packaged game context if possible.
We are also aware of different methods of accessing the RGBA values of the viewport pixels, like writing camera pixels to a render target then reading from it, however it slows down our game too much.