I even tested it in Unreal 4.24 and it was still stretched, so I just made a new animgraph with a single anim node that simply set each bone to what I had recorded for a given pose and that works great.
It is sad that SetRefPoseOverride does not work and no one from Epic responded to my question: Calling SkeletalMeshComponent SetRefPoseOverride on identical instance of a skeletal mesh produces stretched and incorrect bind pose on target - Character & Animation - Epic Developer Community Forums