The raycasts could be done in editor, creating a voxel asset which could then be imported at runtime without any lag.