Dear esteemed RealityCapture developers and friends on the forum,
Greetings!
I am a passionate enthusiast in photogrammetry and 3D reconstruction, regularly utilizing RealityCapture for my projects. The software impresses with its maturity and user-friendliness, and I sincerely appreciate your efforts!
Recently, there has been rapid advancement in 3D Gaussian Splatting (3DGS) technology, showcasing impressive new perspectives in synthesized results while allowing real-time rendering of high-resolution images. This marks a significant milestone in exploration and research. Beyond the original 3DGS technique, the ecosystem under 3DGS, including 3DGS, 2DGS, SuGaR, among others, is thriving. These techniques rely on photos aligned by Structure from Motion (SfM) algorithms and the resulting sparse point clouds. They mostly employ the Colmap open-source alignment algorithm to generate 3DGS or NeRF models for training. However, Colmap’s algorithmic capabilities are not on par with RealityCapture and Metashape, often resulting in issues like slow alignment, poor alignment quality, and numerous outlier points, significantly hindering subsequent technical research.
In this regard, Metashape, due to its open Python API, allows users to create interactive interfaces within the software by writing Python scripts. Users can extract aligned photos, align camera poses, and connect points from the scene, then save them in Colmap’s dataset format. This greatly aids in the technical research within the 3DGS field.
RealityCapture, as the world’s leading 3D reconstruction software, seems absent in this AI research trend. While I understand RealityCapture is a commercial software with its own development roadmap and customer demands, considering the trend of AI development, I hope the RealityCapture development and innovation departments could consider the following requests:
-
Add support for exporting Colmap dataset format: I can provide relevant code and demonstrations. (Benefits: Highlighting RealityCapture’s commercial value and technical capabilities in the AI research field.)
-
Open Python API: Allow users to script interfaces within the software, customizing workflows. (Benefits: Users can address their own needs, facilitating diverse software usage.)
-
An inquiry rather than a request: Does RealityCapture have plans to incorporate machine learning, deep learning, or similar new technologies akin to 3DGS, on top of traditional 3D reconstruction algorithms?
Thank you very much for considering my feedback. Looking forward to further discussions with you all. (If needed, I’m willing to provide my email for in-depth discussions regarding request 1.)
Below are relevant resources:
gaussian-splatting-GitHub Pages
Metashape_Script_Export For 3DGS- Github Pages
colmap-Github
Best regards,
Teacher Du