Solution:
Just add the follow setting in DefaultEngine.ini, and when packaging the game, the cacert.pem file will be staged automatically.
[/Script/Engine.NetworkSettings]
n.VerifyPeer=True
The logic is in /Engine/Source/Programs/AutomationTool/Scripts
/CopyBuildToStagingDirectory.Automation.cs:
https://github.com/EpicGames/UnrealEngine/blob/11ea386451b0beaff2ab34ad3d222def138fcd29/Engine/Source/Programs/AutomationTool/Scripts/CopyBuildToStagingDirectory.Automation.cs#L1204
Why it’s working in editor?
In FSslCertificateManager::BuildRootCertificateArray(), CertificateBundlePath is searched in in the order of ProjectContentDir, EngineContentDir. In editor, EngineContentDir alwarys has the Certificates/ThirdParty/cacert.pem file, so it’s working in editor.
https://github.com/EpicGames/UnrealEngine/blob/c830445187784f1269f43b56f095493a27d5a636/Engine/Source/Runtime/Online/SSL/Private/SslCertificateManager.cpp#L337
How to check it’s really working
On Windows, you can delete all entries in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates\AuthRoot\Certificates\ in regedit, and see if the packaged game really using the cacert.pem file(and not using the system cached one).