Thank you for the update @VictorLerp !
Any ETA for 5.4 Preview release? Even if just as a very draft, please!
For Apple Vision, will we need to compile from an Apple device?
Thanks again!
Great to hear that full immersion (VR) is coming.
While I totally understand why mixed immersive (MR) isn’t coming, it would be cool (and much easier) to implement the following modes:
- Stereoscopic flat game mode
- Portal flat game mode
Stereoscopic flat game mode
This would let you run your UE game in a Window like a stereoscopic 3D movie.
Portal flat game mode
This would let you run your UE game in a Window that lets you see different angles depending on where you are looking.
Technical Details
While it is not possible to use Metal in Immersive Spaces to render 3D scenes in 3D (thus requiring everyone to use the RealityKit rendering engine), it is possible to use Metal in Immersive Spaces to render 3D sceens onto a 2D plane using CAMetalLayer.
Unreal already uses CAMetalLayer to render an Unreal scene in iOS and iPadOS so I don’t think it will take long to adapt this to visionOS. You should be able to use Camera Index Shift to render to different eyes.
For the portal mode, you could get the user’s position through RealityKit and then updates Unreal’s camera appropriately.
Any ETA for 5.4 Preview release? Even if just as a very draft, please!
They are planning to release the first public preview of 5.4 at GDC (Mar 18, 2024).
For Apple Vision, will we need to compile from an Apple device?
Yes you will need an Apple Silicon Mac to compile for Unreal for visionOS. And also to run your game in the Simulator
2 Likes
I have it launching on my device, … it just crashes quickly with:
dyld[498]: Library not loaded: /System/Library/Frameworks/UIKit.framework/UIKit
Referenced from: <CCFFB2DD-E04F-3BF9-A861-8F3EFE1B6C37> /private/var/containers/Bundle/Application/868C641D-4ADB-4F22-86B1-B512DE35552B/VRStarter.app/VRStarter
Reason: tried: '/System/Library/Frameworks/UIKit.framework/UIKit' (wrong platform to load into process)
We’re still working on support, I will update this thread when it’s live on GitHub.
6 Likes
I’ve got my app on the headset, getting it up to the point where it goes past the immersive mode popup into a black void, but it crashes complaining about more than 3 frames in flight.
How did you guys do it?
I’ve managed to build UE 5.4 on my Mac.
Built UnrealEditor and got to compile (pack) the VR Template for VisionOS but at the end I’m having liking problems.
you need open xcode ,and download visionOS 1.0 ,it will success
Yeah, what @MiraTime1 said.
But to elaborate a bit, after you build 5.4, launch the UE editor and create a new project, like one based on one of the VRStarter templates. Make sure to save the project in the GitHub root directory (So UnrealEngine and the project share the same parent). When you regenerate Project Files, your project will be part of the same Xcode workspaces. Launch the Vision OS workspace and you can choose to deploy to the VisionOS simulator. By default you there won’t be a way to add a provisioning profile for the device, so you won’t see the Vision OS device as a target, … But you can get around that by adding IOS platform and viola you can now add a provisioning profile and the Vision Pro will be available as a target.
However… As Victor says though Epic still hasn’t finished their support so that’s about as far as you can get right now. It won’t run on your device yet unfortunately. Looking forward to official support!
Is Mixed Reality something that’s eventually possible in Unreal? I’ve read documents that say that Metal cannot be used for Mixed Reality apps. Unsure how Unity does it…
No. Unreal’s support for visionOS is for VR games/apps only. Unreal could support “Mixed Reality” in some half-hearted way like Unity does, but unless Apple changes the way things work, there is no way for to get full Unreal support working.
Metal can be used in Mixed Reality, however, not the way everyone wants it. You can only render using Metal to a flat 2D plane. See the example tweets I posted above. You cannot use Metal to render in 3D in Mixed Reality.
Unity does not use Metal. There is only one renderer allowed to make 3D mixed reality apps, Apple’s own RealityKit renderer. Unity created a translation layer (called Polyspatial) that translates Unity code to work on top of the RealityKit renderer. Some developers have made something exactly like this for Godot.
And, that situation turns my $4k Vision Pro into a boat anchor.
JK JK, I knew about the limitations in advance and I’ll prototype in Swift and Reality Kit, … but it sure is limiting. What can we do now with UE VR in Vision Pro we couldn’t have already done on much more widely adopted UE desktop VR etc platforms?
And I do look forward to doing some prototyping for full immersion VR on UE once UE is running on VP. It’s infinitely better than nothing :).
That’s the question! But I would change that specific Unreal related question to:
And the answer is… ‘nothing new’, just better hardware (as higher resolution, for example). As much, you could do less 
, due to closed Apple’s ecosystem
That’s great to hear. A few questions on this:
- Is the build process going to make use of PC/Mac remote build?
- What AR features are being implemented/supported for this? For example plane detection, image and object tracking etc
- What render pipeline are you using? I assume as it’s fully immersive it’s metal rather than handing over to the device renderer like Unity is doing?
- Will there be a way to enable any elements of passthrough for a mixed reality experience (even if it’s simply designating an object to have passthrough material applied like with ARKit)?
- Does this mean that there are also plans to address iOS’s ARKit regressions which have limited any kind of Unreal AR application since 4.27?
- Highest resolution displays on a standalone device
- Use Nanite and Lumen on a standalone device
- What render pipeline are you using? I assume as it’s fully immersive it’s metal rather than handing over to the device renderer like Unity is doing?
It’s Metal.
- Will there be a way to enable any elements of passthrough for a mixed reality experience (even if it’s simply designating an object to have passthrough material applied like with ARKit)?
No, Apple has not provided an API to do this so it is not possible.
Does this mean that there are also plans to address iOS’s ARKit regressions which have limited any kind of Unreal AR application since 4.27?
Apple’s visionOS ARKit API and Apple’s iOS ARKit API are quite different, even though they have the same name. Apple really should have named it something different, but they didn’t. Anything to do with visionOS support has nothing to do with iOS ARKit.
Very early Vision Pro device support has landed in ue5-main.
Here’s the commit:
https://github.com/EpicGames/UnrealEngine/commit/12704adb641b9e98a1effaec83230c02f7bd5613
3 Likes
I’ve been checking the repo almost daily since June of last year. This one was incredibly exciting for me, and it’s awesome to see that Alex got it working; I’ll be trying this weekend as well. Lots of work to be done, but I’m really glad Epic finally has a device to test with. Hopefully most of the glaring bugs can get worked out in time for the 5.4 launch.
I’ve been chomping at the bits to use my 10 years of Unreal Engine XR design experience on the Vision Pro… so much potential!
2 Likes
Apple has cancelled the Epic’s developer account, leaving it outside the whole Apple ecosystem…
Will you, Epic, continue this development, for a such tyrant company? Maybe they should loose any VR game made with Unreal, and learn the lesson.
Their account has now been reinstated. But it didn’t affect their ability to develop for VisionOS as the account ban was strictly Epic’s Sweden developer account, which was intended to be used for launching the Epic Games Store on iOS in Europe. This hasn’t affected developers in the United States that are actively working on Vision Pro support for Unreal Engine from what I can tell on the UE5 Main branch on GitHub.
1 Like
Anyone able to successfully build for Vision OS in the 5.4 preview?
No luck here with A8 rendering enabled and Vision OS plugin on, sadly.
build for Vision OS Error :
2024-03-18 21:30:03.882 xcodebuild[4422:45454] Writing error result bundle to /var/folders/13/xsc62m995mz7mzy9xrlmhpbc0000gn/T/ResultBundle_2024-18-03_21-30-0003.xcresult
UATHelper: 打包 (VisionOS): xcodebuild: error: Unable to find a destination matching the provided destination specifier:
UATHelper: 打包 (VisionOS): { generic:1, platform:visionOS }
UATHelper: 打包 (VisionOS): Available destinations for the “MyVisionTest” scheme:
UATHelper: 打包 (VisionOS): { platform:visionOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-xrsimulator:placeholder, name:Any visionOS Simulator Device }
UATHelper: 打包 (VisionOS): ERROR: Failed to finalize the .app with Xcode. Check the log for more information
UATHelper: 打包 (VisionOS): Total time in Parallel executor: 5.21 seconds
PackagingResults: Error: Unable to find a destination matching the provided destination specifier:
PackagingResults: Error: Failed to finalize the .app with Xcode. Check the log for more information
UATHelper: 打包 (VisionOS): Total execution time: 13.19 seconds
UATHelper: 打包 (VisionOS): Took 13.30s to run dotnet, ExitCode=6
UATHelper: 打包 (VisionOS): UnrealBuildTool failed. See log for more details. (/Users/wanglingyun/Library/Logs/Unreal Engine/LocalBuildLogs/UBT-.txt)
UATHelper: 打包 (VisionOS): AutomationTool executed for 0h 0m 15s
UATHelper: 打包 (VisionOS): AutomationTool exiting with ExitCode=6 (6)
UATHelper: 打包 (VisionOS): RunUAT ERROR: AutomationTool was unable to run successfully. Exited with code: 6
PackagingResults: Error: AutomationTool was unable to run successfully. Exited with code: 6
PackagingResults: Error: Unknown Error