Switched from 5.1 to 5.3, camera goes flying somewhere only in shipping builds

Camera is lyra-based, with camera mode, etc. After I switched to 5.3, in half of cases when I try to possess pawn and set camera mode, I get camera hundred meters from the pawn, this only happens half of times in shipping builds, I never got it in dev build or in editor.

Maybe this is related to some code changes, or just garbage collector kills some pointers somewhere.