Hi !
If you need to work with large data sets, I would concentrate on optimizing for memory and performance as much as possible. As such I would rather use a dedicated solution to leverage all available advantages of different properties of voxels (uniform size, grid-based location, solid surfaces, etc.). While there might be some overlap in terms of a high-level approach, the implementations would be tailored.
Have you seen the Voxel Plugin (https://voxelplugin.com/)? I haven’t used it personally, but if you want to kickstart your project or make some initial prototypes, that could be a good starting point 