Buggy Lightmaps

Attempting a hybrid lighting setup on an interior scene but after baking I get wildly buggy looking lightmaps. In the attached my brick wall should be white.
using just 2 lights to simulate window light. An area (Rect) light with Ray Tracing disabled and a Sun (Distant Light) with Ray Tracing (cast shadows ) enabled.

Using default settings to Build my Lightmaps. Attached 2 versions, one with low lightmap resolution and one with resolution increased to 4k.

Can anyone suggest what is happening?