Save video feed from camera object.

Hi!

I have developed a simple object tracker in C++ and successfully send the rotation and translation vectors to a pawn class in UE4. I have connected both pieces of code using shared memory. All good so far :cool:. My next goal is to send the camera view of the camera I have in the UE4 scene to my other C++ application. For this purpose Iā€™m planning to follow the same strategy and make use of shared memory. However, before worrying about the ā€œvideo transfering stageā€ Iā€™d like to understand how the camera system works.

So, instead of trying to solve my problem directly Iā€™d like to first accomplish a simpler goal: gather a few frames from a camera object and save them to disk. This is the first two doubts I have:

  1. Where does the camera object live? GPU or CPU?
  2. Iā€™ve been exploring the ā€œSceneCapture2Dā€ object and do some test inside the editor (displaying the camera feed on a plane). is this a path worth exploring?
  3. Is RHI (Rendering Hardware Interface) the way to go? RHI doc

Any ideas are very welcome.

I think a possible way of making my inital goal work would be to take a screenshot of what the camera sees (Iā€™ll start by just using the viewport) but, instead of saving it to disk, use the TArray<FColor> Bitmap and copy it to a shared memory location in the RAM. Here is how I think I could make this work:

Iā€™ve been checking the pipeline of methods that come from the method call


FScreenshotRequest:RequestSceenshot(<fileName>, <options..>)

And finally reaching


FEditorViewPortClient::ProcessScreenShots(<args>)

Around the middle of this method I belive the line


GetViewportScreenShot(InViewport, Bitmap, CaptureRect)

is the one returning the image array Bitmap Some how I believe I could make this variable accessible from my C++ class.

Would something like this make sense?

Iā€™m working on the same topic right now. Do you succeed in the taking screenshot of the camera view? Howā€™s that work? Thanks!