SOLVED! Just in case someone else is having this issue, I increased the quality of the baking and now the noise is gone. It seems like low samples on the global ilumination affects the volumetric lightmap accuracy…somehow.
