Hey Christ-
I’m wondering if the buttons you are using are translucent? You could try turning on the “Allow Translucent Selection” under the Quick Settings in the toolbar or by pressing T. Hopefully this will allow you to select only the button without selecting the objects behind it.
Cheers