Shadows are not building correctly

Also, from the lightmap it looks like you probably have too mean UV seams, limiting the number of seams will mean that you can utilize the space more and get better results.