I found the cause of the problem. Its somewhere else. It has something to do with the network replication.
I think the server wants to adjust the character rotation and that causes the character jerking. (not the camera is jerking, its the character)