So I found the solution. I looked at everything I changed from going from UE 5.5 to 5.6.
I added the new AMD FSR4 Plugin and thought it was turned off all the way in project settings (I activate it with console commands later if the user chooses it). So here is the screenshot of the project settings where I had “enabled” off already. But: Frame Generation was enabled by default while I thought having FSR4 disabled in general settings above would also turn off frame generation below, regardless if it was checked.
So I learned something new: FSR frame generation was enabled even though I had general FSR turned off above. And the horrible delay was probably using frame gen on a locked 30fps using external frame capping.
I hope if someone has the same problem they will see this post 