And that’s the thing. I am wondering if I can run the data through Marching Cubes, to generate an isosurface. From there, that can be used for collisions, as well as a lighting surface (by setting the V colour component to like 100).
Here is another video or two: