I’m still new at Reality Capture. I’ve gotten into Gaussian Spats using 360 footage from an Insta360 camera. I’m currently using SfM to generate Point Clouds and camera positions with varying results. I’m wondering if I can get camera positions and point cloud data that is cleaner in Reality Capture.
Hi @tomhalpin8, since RS doesn’t import 360 imagery yet, I assume you use some script or third-party app to decompose the 360 into multiple perspective images. Could you elaborate on your workflow more?
Hello @JakubVanko , I’m currently using an app off of Gumroad called 360° Gaussian which can be found here. I just give it 360 equirectangular footage, it extracts the amount of frames I want, as well as generating masks for people, then it uses some version of Shape from Motion (SfM) that works with the 360 footage WITHOUT splitting the frames up into a cubemaps or smaller frames, I think that version is called SphereSFM GUT. Not sure how that part is done. From that it generates a point cloud and 360 camera position. From there I use Lichtfeld GUT Training, allowing the Gaussian Splat to be trained directly from the 360 footage. The splats come out well enough, but the process is still pretty slow, especially with big detailed interiors. The point clouds can be sometimes messy with lots of floaters, making the GS cloudy and lots of clean up needed.
I’m just try to find a nice streamlined way to make awesome Gaussian Splats directly from 360 footage. I’m getting close.
There seems to be a lot of new research generating point clouds from footage near realtime, just nothing that I’ve seen from 360 footage, which is what I’m after since the camera position and point cloud currently take the longest to generate.
I hope this was helpful 
1 Like
Very helpful, thank you . So for now, you can get the camera poses and point cloud in RealityScan, then export in Colmap format and train the GS in Lichtfeld Studio, for example. The only limitation is that you need to feed RS with the cube faces. If you try this workflow, please let us know if you get cleaner results. By any chance, would you be interested in joining our internal Beta program? We do research on this topic.
1 Like
Still new to the workflows, and have yet to try to convert 360 footage to cube face format. I’m definitely interested in joining the internal Beta program! I’ll give the workflow you suggested a try when I get the chance.