I’ve been struggling for some time to find the “right” way to do camera-based AR on PC. What I would like to do is take a camera image from a webcam and draw augmentations on it, while also using the image somehow for processing like object tracking etc… A starting point for me would be to replicate the standard Marker Tracking things that use Vuforia etc. to augment a 3D object onto a tracked location in 3D space.
I know I can do this by for example using OpenCV to get and process the camera image. To create the AR view, I would attach a plane to the camera actor that would be always at the far plane of the camera. This would then have a material which would sample the uploaded camera image in a texture.
However, it feels like with all the facilities that must be in the engine for supporting AR modes on Android/iOS, as well as all the stuff for virtual production, there should be something simpler here?
I can’t for the life of me find the right starting point for doing this properly without hacky half-assed solutions, so does anyone have some suggestions there?