Just for your information when you set the lightmap resolution it’s in pixels… as any other maps, just like texture maps!
So this is your current resolution (256) on your lightmap to let you show how low res it is still… IMO!!!
…and this is your main wall’s lightmap in pixels (!!) again! …just imagine how it would look as a texture map on your mesh… 
The same stands for the lightMAP too!
Also areas that have smaller than 1 pixel information will not get information. The way you keep the uv islands for your wall pieces together (1 inner and 1 outter face) with low res lightmap will generate “random” looks…
You can keep only the visible parts on your lightmap to get a better resolution /those part will be never visible anyway/!
…also I’m not sure if you’re using lightmap compression…