I’ve been working on a cross platform project with UE4 and ARCore / ARKit. Running tests on both platforms, I’ve found that image detection does not work with ARKit and UE4.23 or UE4.24. UE4.22 seems ok, but with an older build of XCode. ARCore seems to be working fine with image detection in 4.22, 4.23 and 4.24.
Has anyone had any success with this feature using ARKit and 4.23 / 4.24 image detection?
Ive tried a handful of tests including new/fresh project using handheld template, adding the candidate image at runtime, and testing multiple image types including QR codes but none of these are successful with ARKit. Surface tracking is fine, and is a temporary solution for what i’m creating, but i would like to be able to get my augmented reality game boards working on iOS as well, so image detection is critical as the desired app evolves.
I’m open to rolling back to 4.22 and rebuilding code since it’s not backward compatible, but if there’s a known issue/fix on the horizon, it would be helpful to know this.
I’ve checked the issues database for bugs relating to this but haven’t found anything. i do see a screen tearing issue with 4.23 and 4.24 ARKit camera applications, but that bug doesn’t address image detection.
Image detection worked for me 4.22, 4.23, 4,24.
The default template’s debug menu shows them via Show Planes. Images are shown as planes with their „friendly name“.
You can test images in XCode, it will evaluate how suitable an image is for tracking.
But apart from that, oh boy… what an ordeal. AR is only for the brave, never had this many issues for desktop or VR projects.
Using 4.22 on MacOS comes with more problems, some things to watch out for:
if you follow the setup guide of the documentation, the camera image looks dark and gritty. That’s a bug. MobileHDR has to be enabled in the projects render settings. Affects all version from 4.22 to latest release.
With 4.22 on Mac, make sure the path to your project has no empty space.
It mistakes an empty space as the end of the path and the rest of the path as an argument.
So if your project is in „/user/Unreal Projects/Projectname“ , the log will give warnings, „/user/Unreal“ does not exist and „Projects/Projectname“ is an invalid argument. This can also affect DDC and dragndrop import of assets among other things.
Save often, occasional crashes when moving folders in the content browser.
If you use XCode10 and your ios device is up to date, you cannot deploy to the device. So you need to use latest XCode to deploy an .ipa file.
in case you have Visual Studio for Mac and newer Mono installed, you need to uninstall, because otherwise the UE4Editor does not find any provisioning profiles or certificates, also manually adding these does not work.
command+alt+esc will become part of your workflow with UE4.22. (I thought that is more of a windows thing.)
4.22 often gets stuck at launch (splash screen) when compiling shaders. If users seek out help about this issue they are usually told to just wait until compiling finishes, but no, the editor does not eventually open.
To find out whether it really is compiling or stuck, you can hear the fans spinning slower (this sounds so archaic) and then check MacOS Activity Monitor if there are Shadercompiler at work. If not, it finished but the editor did not notice. You could wait years with nothing happening. Kill UE4 with command+alt+esc and start again.
if you use 4.22, you have to use XCode10. Whenever you start 4.23 or later you have to remember to switch via terminal command XCode version shares will recompile for quite a while, because different XCode version. If you forget, kill UE4 command+alt+esc and open again, because the UE4 splash screen has no abort/cancel button.
Github shows big code changes for ARKit in 4.25, let’s hope for the best. But so far every release also came with new bugs. There is obviously no testing process going on. Image detection is currently broken for ARCore in 4.24 Unreal Engine Issues and Bug Tracker (UE-87102)
So there is a chance of 4.25 with fixed ARkit camera but broken ARCore image recognition. It just never ends…
Bugs can always happen, but constantly breaking features for over a full year now is something else.
ARKit and ARCore are evidently in experimental status. Is it marketing pressuring it as usable in production? This hurts the reputation of an otherwise great engine.
this helped me troubleshoot a lot going on between engines and platforms, but i’m still having issues with iOS image detection on 4.23/4.24. which tutorial did you use for image detection?
i used this one: ARKit 1.5 Image Detection in UE4 4.20 | by Joe Graf | Medium
i actually had to use 4.24 for iOS anyhow because of the “Truedepth API” bug that causes auto rejection in 4.23 when not in use. so for android i’m using 4.23 to avoid ARCore blackscreen on initial load bug, and 4.24 for iOS Truedepth API submission bug.
i checked out the hotfix notes for 4.24 but didn’t see anything related to ARKit / ARCore.
yes, the article by joe graf is the only source of information for the setup.
Are you using the same ARCadidateImage and ARSessionConfig classes for both (ios and android) or the ARCore-specific classes maybe?
Some people hear ARCore is cross platform and use it for ios too, with bad results.
I am not using the blueprint example from the article. That is the reason i recommend using the ShowPlanes in the AR template debug menu, because it is independent from any later logic.
Also Casting every tick in the article is not efficient. There is a GetAllARTrackedImages function, no need to cast anything.
Some print string and debug draw first might help. I use it with a timer by event.
Check ios/firmware, Is the device really capable of image tracking? Use blueprint: IsARSessionTypeSupported, set to Image, it should return true.
for reasons beyond me, the image detection does seem to be working now on iOS. have no clue why it wasn’t working before, but now it is. however, (and i’m guessing this is my own error), the image scale detection is returning different values between ios and android. android seems to be right on point with scale, but iOS seems to have a much larger scale associated with the same image. this is using the same image candidate for both platforms, so not sure what’s going on here, but will add more info as the troubleshooting progress.
