At least this answers why my buttons seemed broken. I thought I did something wrong or forgot something. Turns out I have use mouse for touch enabled. I guess for now I will assume the button is fine and move on then. Maybe we can get a fix for this in a future update or maybe a warning when enabling the use mouse for touch option.