Mixed Reality Capture Feedback for Epic

[malcriado]

@eh.cin

We went to an agriculture exhibition with MRC and didn’t had any real problems with it. Calibration didn’t work against the green screen again, but we were able to circumvent that (read the earlier post).

We used MRC to bring a real field (about 2 hectares or 5 acres in size) to the showroom. A 3D model of the field was made from drone images (hence the bad resolution). We also had lots of data from past summer, including index layers (NDVI) from satellite images, a 3D model of barley made from a real plant (with photogrammetry technique) and some farm machinery (from manufacturers’ CAD-models, converted with Unreal Datasmith). We also had a real weather station in our virtual field in the presentation and a real (green!) plant and managed to keep it visible in the videofeed. We used real data and models made from real things almost exclusively, nothing major was made by artists. Videofeed in the big screen had two sources: mixed reality using the external camera and presentators view from the headset (with virtual hands). Director switched between those two video sources based on what was happening in the presentation.

The show started with a small introduction speech made by the presenter (industry specialist) and then he literally walked to the virtual field and put headset on. The presenter presented lots of information about cultivation methods and results from past summer. Viewers were mainly farmers and we used Mixed Reality as a presentation tool, not a plaything. That large screen was visible through three exhibition halls and worked as a lure to bring people to watch some educational content. The pillar of earth shown in the image is actually a hole in the ground of the real field (there was four holes/pillars in total). We just inverted the basic idea and used real images from the hole as texture for a pillar that ascends from the ground. There was also a panel discussion at the end where participants were standing in the virtual field. A single presentation lasted about 40 to 50 minutes and we ran it total of 10 times during three exhibition days. There was no issues related to Unreal Engine or MRC. We had to scale our presentation down (to one person and one camera), because we fried one of our PCs, but that was just a hardware issue.

EDIT: The carpet was standard cheap and thin exhibition carpet in light green color and glued to the floor as usual, nothing special. We had exactly same carpet in other areas, but in different colors. We had tested a small piece of a carpet at the office in advance. It seems that the MRC works well even with a carpet that has some grooves.

This is what happens when Unreal Engine, Unreal Datasmith and MRC are used outside of the gaming context. Thank you Epic for great tools you made available, this was a fun project!

eh.cin very nice work! could you expand how did you manage to make it work? thanks in advance!