There are multiple solutions to this problem. I ended up covering my model with a cover mesh. That way the corners (and the model of the building in general) isn’t directly exposed to the lighting. You can use a very low LM resolution to use a very low amount of resources.
You could also snap the lightmap UV’s to the grid of the corresponding resolution. (https://www.reddit.com/r/UE4Devs/comments/246whl/the_most_important_thing_about_lightmaps/?st=jfffzf8t&sh=515252d5)