Complex interior architecture effort

Hello, I am working on an interesting project and wanted to make forum post about it, in the hope a few questions can be answered.

I have been commissioned to market a luxury property in Los Angeles, a penthouse in the Ritz Carlton. As part of our services, I have been experimenting with offering VR renderings for Vive. Our goal is to create a photoreal VR area of about 1200 sq. ft, that the user can teleport around and view in VR. We’ve captured a really high quality skybox with our drone, at about eye level to the unit, and now we’re working on creating the interior rendering.

Our first run there, I captured about 450 test photos in a small area (Nikon D810 with 10-24 lens), as well as completing a Matterport scan of the full interior (along with some video etc).

The first pass through Reality Capture went well, but there were some huge mirrors that caused trouble. So, we returned with a roll of brown butcher paper, covered all the mirrors completely, cleared the unit as much as possible, and captured a final batch of ~1000 photos of this small space.

These gave a much better result in RC, the textures are incredible. When you stand in the space in Vive the effect of the natural light is really amazing. Only one big problem: the flat planes were still looking really poor. In the end, we began the process of modeling the planar surfaces of the unit as a new mesh. So in the end, I now have 3 models, all perfectly scaled and aligned:

http://imgur.com/a/RNlMq

So, here is my question, how can I reproject the textures onto the cleaned up mesh? I’ve been aligning and editing the meshes in Autodesk Remake. I tried to pull the clean planar OBJ into RC but it was aligned incorrectly and all attempts to correct it and export it proved inconclusive. Once the mesh is perfectly cleaned with all planes from the RC mesh deleted and replaced with hand modeled planes AND supplemented with Matterport geometry for distant areas, how do I align the mesh correctly to reimport back into RC? As far as I can see there is no way to re-position the corrected mesh once imported into RC, so I gather that the corrected meshes need to be set to some coordinate plane fed by the original RC export? Or can I set the coordinates in RC BEFORE I export the photogram mesh?

Reality Capture is really incredible, a huge achievement. I look forward to using it to get my clean mesh textured…just feel I’m missing something crucial in trying to align my RC export and my cleaned-up mesh. Not even sure this reprojection is possible in RC. Has anyone else tried tackling complex interior architecture capture?

Any insight is appreciated!

Hi Cole
The main issue when you process data in an external application is that it changes units ( form meters to inches or cms ) so the scale is not correct. This can be solved quite easily when you use proper units so that the scale is NOT changed. ( do some testing on a small triangle count model for faster work )

If the position is not correct ( any screenshots ?? ), then it looks like the model has been moved in post-production. This can be solved with right workflows to avoid model movement.

When you have already finished all the post-processing, then there is only one way to solve this. You need to export a low poly model from RC and ALIGN the edited model to RC model in an external application, say Geomagic or Meshlab etc… So you put a post-processed model in the same position, orientation and scale as in the RC model.

Hi, Any update on this project. Cole I sent you a private message.

Hey Cole,

Interesting stuff you’re working on! I have been in the preliminary phase of doing this, combining exterior aerial UAV data with interior scans using DSLR’S. The addition of the Matterport camera really peaked my interest, I’d love to know your workflow with it.

Unfortunately I have made no where near the progress you have with this, but I may have found a workflow guide that solves your problem here. The guys at VALVE/Steam (HTC’s partner in the VIVE) are posting workflow guides creating VR content using RealityCapture content as inputs. Some of these guides are the most comprehensive that I’ve found so far on creating hyper-realisitic VR recreations from photos. They have an “Advanced Interior Photogrammetry” Guide that hits your export/modify/re-import problem on the head.

If you haven’t solved/seen this already, check out:
https://developer.valvesoftware.com/wik … ogrammetry
https://developer.valvesoftware.com/wiki/Destinations

Best of luck!

Thanks for the links Peter, really interesting stuff, surprised I hadn’t stumbled on this before!