Lightmap UV's Shifting After Bake

I make my own lightmaps UV’s so that I can snap them to the grid to avoid black edges. Pretty standard, I keep my UV’s on the second channel on export, disable generate UV’s on import, and set the lightmap res and minimum lightmap res in the object properties.

However, I noticed that my UV’s get shifted around after I build lighting. If I override the light-map resolution to the same res, it snaps back to how its supposed to be, but once I rebuild the lighting the UV’s shift back. Any idea what’s causing this?

Before Building light:


After building Lights


Here you see the Lightmap UVs before and after Bake. They shift slightly. This produces black seems.