The vision build is made with UE_USE_SWIFT_UI_MAIN at true, built 5.5 from source.
Steps to Reproduce
Hello,
When trying to activate a textfield in UE 5.5 using a Vision Pro I get a crash in FIOSPlatformTextField::ShowVirtualKeyboard because [IOSAppDelegate GetDelegate].IOSView is nullptr.
I tried to disable the code that checks the value of View->bIsUsingIntegratedKeyboard, and acted as if it was always false so I don’t read the View pointer. that is how I did it:
void FIOSPlatformTextField::ShowVirtualKeyboard(bool bShow, int32 UserIndex, TSharedPtr<IVirtualKeyboardEntry> TextEntryWidget)
{
#if !UE_USE_SWIFT_UI_MAIN
FIOSView* View = [IOSAppDelegate GetDelegate].IOSView;
if (View->bIsUsingIntegratedKeyboard)
{
if (bShow)
{
FKeyboardConfig KeyboardConfig;
GetKeyboardConfig(TextEntryWidget, KeyboardConfig);
[View ActivateKeyboard:false keyboardConfig:KeyboardConfig];
}
else
{
[View DeactivateKeyboard];
}
}
else
#endif
I don’t crash anymore but the alert window with the textfield doesn’t appear. Is this because the Alert is UIKit and Unreal on VisionPro is SwiftUI and you can’t just mix the two like that ?
I am currently looking on how to create a swift alert window using the UE/Swift bridge, but maybe there is a better way of inputing text on visionOS ?
Hello, sorry for the long delay. I started trying to look at this, but ran into some blocking issues.
Unfortunately I don’t know a lot about this. I think it is possible to use UIKit in visionos, there is some compatibility between SwiftUI and UIKit. However I am not sure we are configured properly to actually make use of it, and I’m not sure if this specific feature falls within the compatibility. I think there is a SwiftUI virtual keyboard as well which might work if the UIKit one can not.
Debugging into @implementation SlateTextField show might be informative.
Ah, so you put it into the swift ‘splash screen’ that comes up before we go into immersive/mixed? Yea I think that is a good way to do ‘initial settings’.
From looking at the keyboard pop up code in unreal for ipad it looks to me like we are hooking it in in a way where the system keyboard pops up, but the textfield, etc are all rendered by unreal. So probably it is possible to do that in visionOS, but that setup is fairly complex and I’m not exactly what needs to change for it to work.
Oh, and there was some improved support for making that SwiftMain file more elaborate in UE 5.6:
https://github.com/EpicGames/UnrealEngine/commit/c649151ab0cd530b3ec407457426ebade0b61775
Ok. If you do every have problems with the timer-poll solution we did implement some improved ue->swift and swift->ue communication in 5.6 and 5.7 which ought to be discoverable by examining SwiftMain.
Hello, to fix this we went a very different way.
Where we open/close the window we use the existing bridge between Unreal Engine and swiftUI to get some data from Unreal to swift. That way we can signal swift it should open an alert window with a text field. And then we can send the result from that alert window back to Unreal’s Textfield.
From my understanding it would be hard to use the compatibility between UIKit and SwiftUI here. Because we still need a swiftUI window to render UIKit content.
So the most straight forward thing was to re-create the alert window in swift.
I didn’t use the initial “splash screen” instead I created an invisible “volume window” in swift that stays alive as long as the game runs. And in that volume window I run a timer that polls Unreal every 0.1s. When it receives information that a textfield needs to be opened I open a new 2D swiftUI window that contains a text input field, and a button to cancel and validate (technically I use a swiftui alert window, but the idea is the same).
This way when I activate the swiftUI textfield, VisionOS handles opening the default Vision OS keyboard. Once I click on “accept” on my alert window I send the content of the textfield to Unreal and close the swiftui alert window.
None of it is super clean. But it works well enough for the needs of our game.
You can close the question and mirror it to the public forums if you wish. We have a solution that works for us.