The VM’s draw from that key vault each time they are created, so if you can update it there, it should be okay. However, we have not had any coverage on that for testing, so I can’t guarantee that right now.
Can you set Idle to 0 in the dashboard? If not, the way to spin down all instances is to go into the VMSS in each region, go to Scaling, and set your number manually there. But that also disables scaling up. We may have not accounted for truly “on-demand” usage since the wait time to spin up one single instance is quite long.
As for other errors in the dashboard, the last time I tested that feature was in June. I will put it on my list to look into.