I am not sure what is wrong but maybe it would help to tell it not to set actor rotation unless the difference between current rotation and the one indicated by your angle calculations is bigger than 2 degrees.
I have a similar system for my game but mine does an interpolation toward the calculated angle instead of changing instantly so maybe that has something to do with it.