Your setup seems reasonable. Our projects are pure C++, or as close as possible, with very little blueprint use except the HUD widgets.
One issue - are you able to control more than one type of pawn/character? If so, might want to move inventory down to character.
Another - do you have a player controller state stack? We have one for stacking player states like Walking >> ExitMenu >> Paused >> CutScene >> etc. Each state determines what inputs do.
Finally, are you using a dedicated HUD class? Each map allows you to specify default - GameMode, HUD, PlayerController, and Character.