Finally taking the leap into making a professional version of our art installation, “The Sandbox of Life” and I have some questions about the most compatible 3D depth video camera for use inside of Unreal Engine.
The piece is one of those augmented reality sandboxes and the first iteration was in 2013 using the original Kinect.
It essentially reads the height map of the sand below and then makes cool visuals and landscapes based on that which are projected in real time right on top. Super cool science center type stuff.
Since then we have used various cameras like the Kinect V2 and also lately the Intel RealSense D435.
Right now we are using the Processing IDE to make the visuals but it has been limiting in power and there are very few developers.
So, the core of this ask is “What is the best 3D depth camera for use in the UnrealEngine?”
After much research we cannot find many answers to this online and are hoping to get some tips or reports of direct experience.
For instance, does the Intel RealSense D435 work? It kind of seems like there is an SDK but it’s only for UnrealEngine 4. Can this be ported to 5?
How about any of the new cameras from Oak D or even Orbec Astra?
Do the cameras need an official plugin/SDK or anything like that or can UnrealEngine just accept any USB webcam type feed from any connected camera kind of like how Zoom or Skype do?
Ideally we want a camera that is still being manufactured and one that gives us a greyscale depth map. We don’t need skeleton or object tracking. Just depth.
Alternatively, would it be better to build a simple app in Processing that can read the camera and then stream that into UnrealEngine over Spout or NDI to then do cool things with it in Unreal?
Overall we want to make an eco-system simulator for educational purposes.
Thank you for any insight we look forward to learning more about this!
