So, there is no “fix” for this, as far as I could find. There is only a workaround.
The workaround is to export the animation and model from UE4 and then import it into blender, fix the odd twitches there (which weren’t there in the original file) and then re-export. This works fine. But its really annoying. Hope someone, can find a better way to fix this.
This model was exported from UE4 (armature included) and the entire leg was rotating weird from the hip when I imported back to UE4. Stuff like this happens in blender too if you pass the 180 degrees limit for a bone that’s used for IK. Not sure why sometimes it only happens in UE4 and not blender.