I only had this issue when one of the monitors was controlled by a much weaker gpu or directly trough mobo.
running 2 4k, 1 2k, and a graphics tablet on 1080p without any hitches on a 1080ti now, so your setup on a 1080 should run fairly well.
Id blame win10 partly for that.