[UE4.27] Meshes look better before Building lighting?

I have meshes that look worse with Built (Production-quality) Lighting.

I’m not sure what the issue is here. Is it the lightmaps? Post-processing? Does it have anything to do with overlapping UVs? Did I do something wrong while building the models?

What can I do to get this closer to what it looks like pre-Build? I’d appreciate the help. Thanks!

EDIT: It seems to have something to do with double-sided walls, as removing the bricks from one side fixes it. Why is this?

Same as above, left only has one side of bricks, right has both sides.