Depending on how you UV map them, the lightmap could bleed onto the visible area if you have those sides capped