Unreal Engine 5.1 Preview

You have to click the + and then search for the preview 5.1.0 as an option of the version you wish to install. If it’s not there, then yes, you need to logout and back in.