So I found the answer you need.
[MobileSlateUI]
bTouchFallbackToMouse=false
This will go inside of the DefaultEngine.ini file that is located in /Config/DefaultEngine.ini
But this seems to only work for me when I am either in a packaged build or a standalone game not sure where I need to put it yet so the games run in the editor also work.
Edit:
This sort of fixes this issue. I have found that some things stop working because of how the input system works in unreal. What this setting does is makes it so unreal engine no longer calls the mouse versions of the input after the input returns FReply::Unhandled. That causes some issues in places like buttons inside of scroll boxes not being able to scroll anymore due to the SViewport capturing the mouse cursor instead of the button getting the mouse input. So while this may fix your issue where touch on a button gets called twice it may introduce other issues to your code.