Don’t bind the visibility, just get your button event OnClicked and inside it get the the panel visibility, add a switch on it, and do what you want based on the visibility:
OnClicked
Add what ever combination you need.