Collision box, whatever the widget is interacting with will tell me the details of the widget that’s interacting.
I don’t think this is doable with 2d widgets and text box menus.
Colour match, I have a colour matcher that waits for a colour overlay and then tells us what the overlay is.
check how far the animation moved and mathematically work out which answer it is.
This is an absolute nightmare of maths and systems.
I think colour matching is the way to go.
Just trying to figure out how to show the colour of a pixel?