Okay, so the problem was the camera collision test, I guess it just ignores colliders up to the pawn root.