Displacement results not as desired

I am a beginner still in UE4 and Blender. One Blender to UE4 house asset was fantastic, but it used parallel occlusion. The SUV addon for Blender in that course was very, very good for me to deal with correct texturing. Anyway, I am stuck with trying to get displacement correctly working for bricks and roof tiles.

Is it better to separate the roof from the house?

I watched many youtube tutorials, 1 course Blender, 1 course UE4 and one combined course. For this part stuck. If I understand this, I could move on to create all buildings correctly and move on to decal. Help would be very much appreciated.