How to disable collisions so that the mouse cursor isn't affected

I currently have some code set up so that my player always faces the direction of the cursor, this works great apart from sometimes when I move it over objects in the scene (With no camera collision) it bugs out and flicks around. Am I changing the wrong part in my collision settings?
collision