I mis-remembered slightly – it’s not the state machine that has a sub-state-machine, it’s the animation blueprint.
So, this is how you’d set it up:
Right-click the animation blueprint animation graph, and type “new state” to find the “new state machine” option.