I am working on an open-source UAV simulation where I am trying to capture and save images from a camera that’s attached to the drone. The image underneath gives an idea of how the game looks like. The rightmost view on the bottom is the actual view from the camera, the other two are just processed versions of the same image.
Right now the way it’s set up in this way: There’s a camera asset, which is read through the code as a capture component. The three views in the screenshot are linked to this capture component. The views are streamed without any problem as the drone is flown around in-game. But when it comes to recording screenshots, the current code sets up a TextureRenderTargetResource from this capture component, and subsequently calls ReadPixels and saves that data as an image. For some reason, this is slowing down the whole game by a lot: drops from ~120 FPS to less than 10 FPS when I start recording.
Looking at this question and this related article, they mention that ReadPixels() “will block the game thread until the rendering thread has caught up”. The article contains sample code for a ‘non-blocking’ method of reading pixels, but it doesn’t work for me yet. Are there any more efficient methods that can achieve what I am trying to do? I am a little confused as to why the code has no trouble streaming those images on-screen, but saving the images seems way more complicated.
I could be wrong here, but I believe that the reason this is so slow is because in order to save the data, you have to read back from the renderer / GPU back to the CPU. That’s a slow operation no matter what and isn’t the fault of Unreal.
When you say “saving” screenshots - do mean writing them to disk? If so that’s not something you want to do on the game thread anyway, since you’ll be limited on speed by the end users hard drive ability. If that’s something you want to do in realtime, you need to spin up a new thread to do it or risk a lot of stalling. No different to saving a game really.
I haven’t checked but the non-blocking method of reading pixels probably means you’ll get out-of-date data when you save it - so I don’t think that’s a solution for you anyway. There are existing methods of exporting a render target to a file - and presumably the render target is stored in RAM somewhere prior to being sent to the GPU - so you could memcopy it somewhere and spin up an async thread to save it to disk maybe.
Thanks for the reply! Yes, I am interested in saving images to disk, something that would show me later what the drone’s point of view was like. Would you perhaps be able to point to any resources I can start off with for writing an asynchronous thread that can do this? I am not too concerned about the frequency at which the images are saved: it can be as low as 10-15 images per second, depending on the time taken for reading the image out, saving to disk etc: but I just want to do it in the background in a way that it doesn’t slow down the gameplay, and every time the image saving procedure is triggered, it should try to save the most recent view, instead of accumulating some kind of a delay.
This is pretty complex code though. As TheJamsh says, you need to read back from the graphics card, so you can’t do this properly if you block the game thread each time. So what the code is doing is basically using multiple buffers in graphics memory to store frames, then asynchronously reading them back to regular memory, at which point the buffer in graphics card memory can be reused for another frame.
I managed to pull out the necessary code and reuse it for an experiment a while back, without really understanding anything much of how it works.
