This turned out to be a weird artifact of my bone constraint bleeding over from blender some how. In my animation in blender, the bones were locked into position, but when putting them in an aim offset in unreal would make the constraint issue become apparent. The trick was just to redo my animations so that the bone constraints were no longer pulling on the hips/feet of my armature.