I am trying to take a 4K 360 30fps stereo video from UE4 projects (cinematic scenes made with Matinee or Sequencer).
I tried Stereo Panorama Tool plugin (Kite & Lighting), but it failed to render properly some problematic parts of some scenes and it’s claimed (by other developers) that it works not good with full scene reflection.
There’s a tool developed by Nvidia called Ansel that can take in game 360 stereo image screenshot. Also there’s a plugin for UE4 that integrates this tool using AnselSDK.
One of the problems is that Ansel can be activated (capturing itself, its session can be activated from code) only by pressing on Ansel UI button (whole Ansel UI injected into the game itself once the session is activated). The only way that I have found to automate the process is to inject Windows keyboard events (mouse events cause rotation of the camera sometimes and instability of the captured images) using Windows API (Native, Winform), AutoIt, etc. (all the tools, techniques are free, didn’t try commercial products) in order to press on required buttons by TABing (menu navigation) with proper time stamps.
The problem with this approach is that the Ansel UI which is injected into the game (UE4 standalone mode) doesn’t catch these event sometimes which causes a lot of problems (missed frames and wrong selection in the menu).
Another problem with Ansel is that if it runs for a long time (depending on a machine), it crashes UE4 itself. So the process should be started again and continue from the point of fail. For this problem I didn’t find a proper solution as well, I would like to use it as Kite & Lighting.
Hope you have some thoughts