Vr third person

Hi I am working on a VR project. I have a character, camera boom and a camera. I have use controller yaw checked on the character, so that I can control character with thumbstick. When I look right or left, the character slides right or left. I am always facing the exact back of my character since it turns with my head. Has anyone else dealt with this issue? Thanks