I have a camera open rendering to a media texture. I’m guessing under the hood it’s an OES_EXTERNAL_TEXTURE being updated with the camera feed and unreal reads it.
I want to also send this camera data into another destination that will do some AR ml related things on it. It might even be running on its own OpenGL context. This would all be with a bunch of C++, if I can just find the right APIs.
Normally I’d do this by having one OpenGL/Vulkan context render something out to a SurfaceTexture and then the other side reads from the SurfaceTexture by loading it as an OES_EXTERNAL_TEXTURE. That way all the data can stay on the GPU without any super slow GPU->CPU->GPU copies. It does require 2 copies but it’s way faster than involving the CPU.
Alternatively if I get access to whatever Vulkan or OpenGL texture id unreal is using, I can write a bit of C++ to do it.
I’ve been able to do this with Unity before, but so far I find no references to getting at the data in Unreal.
Currently the slow way I can think of to do this is to render the media texture into a render target, and then read pixels from the render target to the CPU. I know with OES_EXTERNAL_TEXTURE you can’t do things like glReadPixels.