Deactivate Player Controller while displaying menu?

Is there a way to do this without setting input to UI only?

If I set input mode to UI only, whatever key I was holding down stays held down (i.e. if I was walking, I continue walking).