If someone finds this. I had to do this:
IConsoleVariable* EnableGamepadPlatformCursor = IConsoleManager::Get().FindConsoleVariable(TEXT("CommonInput.EnableGamepadPlatformCursor"));
if (EnableGamepadPlatformCursor != nullptr)
EnableGamepadPlatformCursor->Set(true);
This will set the EnableGamepadPlatformCursor CVar to true. This changes the internal handling of cursor widget. It does not make sense and maybe is a bug, but it works for me.
Then we use SetUseSoftwareCursorWidgets to hide the cursor.