UE5.1 records replays in the development built game, but not in shipping

Do you have it set to your game instance in the project settings?