Assertion failed: result == NV_ENC_SUCCESS

It looks like your problem is related to the bug listed here: Unreal Engine Issues and Bug Tracker (UE-86150)

From the developer notes:

You can install/upgrade the NVIDIA driver using the guide at Install NVIDIA drivers on Windows instances - Amazon Elastic Compute Cloud or build a new instance using the ami “Microsoft Windows Server 2016 with NVIDIA GRID Driver” by Amazon Web Services