After comparing two projects side by side I figured that a different setting in screenspace reflections is at least partially to blame for the FPS drop....