Editor plugin - Keyboard binding not working

I am having the same problem. I am using UE 4.9

I created a New Plugin and chose the Toolbar Button option.

The button works fine, I can even set the keyboard shortcut to the command, but nothing happens when I press the shortcut binding. When I hover over the button it will read out the shortcut binding, but it never works.

Is this normal for the Toolbar Button plugin template? I would have expected it to work by default. Maybe there is an extra step for having the keyboard shortcut fire the command?