I had this problem with a project that needed to stabilize movement from an external controller, and fixed it by writing new versions of the weighted average functions that properly understand the circular nature of angles. The source is available here. I highly recommend them for anyone trying to smooth rotators.
// The shortest distance around the circle from one angle to another. The sign of the result
// indicates the direction, and the result can be added to FromAngle to get to ToAngle,
// be sure to normalize after.
float UWorkbenchMath::AngleDistance(float FromAngle, float ToAngle)
{
FromAngle = NormalizeAngle(FromAngle);
ToAngle = NormalizeAngle(ToAngle);
float Dist = ToAngle - FromAngle;
if (Dist >= 180.f)
{
return Dist - 360.f;
}
else if (Dist < -180.f)
{
return Dist + 360.f;
}
else
{
return Dist;
}
}