Proper Way to do Camera-based AR with a Webcam

Development Platform & Builds XR Development
1

I’ve been struggling for some time to find the “right” way to do camera-based AR on PC. What I would like to do is take a camera image from a webcam and draw augmentations on it, while also using the image somehow for processing like object tracking etc… A starting point for me would be to replicate the standard Marker Tracking things that use Vuforia etc. to augment a 3D object onto a tracked location in 3D space.

I know I can do this by for example using OpenCV to get and process the camera image. To create the AR view, I would attach a plane to the camera actor that would be always at the far plane of the camera. This would then have a material which would sample the uploaded camera image in a texture.

However, it feels like with all the facilities that must be in the engine for supporting AR modes on Android/iOS, as well as all the stuff for virtual production, there should be something simpler here?

I can’t for the life of me find the right starting point for doing this properly without hacky half-■■■■■ solutions, so does anyone have some suggestions there?

2

Hi, you’re right, using mobile AR can be a better solution. No need for markers.
A PC webcam is bound to a cable or built in a monitor. A smart phone or tablet can take care of all tracking AND transmit the camera image wirelessly, that’s another plus.

Suggestion:
If you want to capture the image from the device, just make an AR project and look for a way to stream the content to a pc.
(it’s easy on ios devices, i am not very familiar with android)