Augmented Reality for UE4

great update would come just in time for me; nice to see the barcode markers too. Cheers

Hi There,

A couple co-workers and I are having a minor issue while using the plugin.
So recently we updated our project to 4.16.3 and we have your latest plugin that supports 4.16.
Which has been working perfectly fine for us until this point.
So far we have successfully compiled our projects for iOS and Android.
But for some odd reason when we choose to just ā€˜play level in a new window that runs in its own processā€™
the window for the game crashes upon .
I was thinking that it might have been because my work computer doesnā€™t have a webcam attached to it.
However my other co-worker does have a webcam attached to his and we both receive the same errors when our game crashes.
I copied and pasted the errors below for you to take a look at when you get the .
Iā€™ve noticed that it so far crashes upon enter the init functions in ARToolKitDevice and ARToolkitComponent.

LoginId:b7064cc74b91bd53776972b5e052fb5a
EpicAccountId:

Unknown exception - code c06d007e (first/second not available)

KERNELBASE
UE4Editor_ARToolkitPlugin_4235!__delayLoadHelper2() [f:\dd\vctools\delayimp\delayhlp.cpp:323]
UE4Editor_ARToolkitPlugin_4235!_tailMerge_ARvideo_dll()
UE4Editor_ARToolkitPlugin_4235!FARToolkitDevice::Init() [d:\unreal projects\redbullar 4.16\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitdevice.cpp:539]
UE4Editor_ARToolkitPlugin_4235!UARToolkitComponent::Init() [d:\unreal projects\redbullar 4.16\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitcomponent.cpp:169]
UE4Editor_ARToolkitPlugin_4235!UARToolkitComponent::execInit()
UE4Editor_CoreUObject!UFunction::Invoke() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\coreuobject\private\uobject\class.cpp:4461]
UE4Editor_CoreUObject!UObject::CallFunction() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\coreuobject\private\uobject\scriptcore.cpp:721]
UE4Editor_CoreUObject!UObject::ProcessContextOpcode() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\coreuobject\private\uobject\scriptcore.cpp:2165]
UE4Editor_CoreUObject!UObject::execLetBool() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\coreuobject\private\uobject\scriptcore.cpp:2067]
UE4Editor_CoreUObject!UObject::ProcessInternal() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\coreuobject\private\uobject\scriptcore.cpp:943]
UE4Editor_CoreUObject!UObject::CallFunction() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\coreuobject\private\uobject\scriptcore.cpp:845]
UE4Editor_CoreUObject!UObject::ProcessInternal() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\coreuobject\private\uobject\scriptcore.cpp:943]
UE4Editor_CoreUObject!UFunction::Invoke() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\coreuobject\private\uobject\class.cpp:4461]
UE4Editor_CoreUObject!UObject::ProcessEvent() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\coreuobject\private\uobject\scriptcore.cpp:1313]
UE4Editor_Engine!AActor::ProcessEvent() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\actor.cpp:695]
UE4Editor_Engine!AActor::ReceiveBeginPlay()
UE4Editor_Engine!AActor::BeginPlay() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\actor.cpp:3136]
UE4Editor_Engine!AActor::DispatchBeginPlay() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\actor.cpp:3104]
UE4Editor_Engine!AWorldSettings::NotifyBeginPlay() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\worldsettings.cpp:187]
UE4Editor_Engine!AGameMode::HandleMatchHasStarted() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\gamemode.cpp:229]
UE4Editor_Engine!AGameMode::SetMatchState() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\gamemode.cpp:341]
UE4Editor_Engine!AGameMode::StartMatch() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\gamemode.cpp:206]
UE4Editor_Engine!UWorld::BeginPlay() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\world.cpp:3443]
UE4Editor_Engine!UEngine::LoadMap() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\unrealengine.cpp:10299]
UE4Editor_Engine!UEngine::Browse() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\unrealengine.cpp:9563]
UE4Editor_Engine!UGameInstance::StartGameInstance() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\engine\private\gameinstance.cpp:421]
UE4Editor!FEngineLoop::Init() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\private\launchengineloop.cpp:2613]
UE4Editor!GuardedMain() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\private.cpp:153]
UE4Editor!GuardedMainWrapper() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\private\windows\launchwindows.cpp:134]
UE4Editor!WinMain() [d:\build++ue4+release-4.16+compile\sync\engine\source\runtime\private\windows\launchwindows.cpp:210]
UE4Editor!__scrt_common_main_seh() [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:253]
kernel32
ntdll

I donā€™t understand it but very cool looking !

4.17 update is available to download at unreal4ar.com.

You can download it from your account. If you donā€™t have an account please send me your order id # to info@unreal4ar.com, and Iā€™ll send you a new link.

Thats awesome thanks!!!

;)Thanks ,this plug is very useful.
But I have some issue in use this plug ,I donā€™t know it is bug or something.

When I use ARToolkit GetMarker, I canā€™t use WidgetInteraction to use world space widget. Could you help me fix this problem? I donā€™t have idea how to fix it.

Whatā€™s the difference between this plugin and Appleā€™s ARkit ?

Three differences:

  1. ARKit uses markerless SLAM tracking , ARToolkit (the sdk used in UNREAL4AR) uses black and white and image markers.
  2. ARKit is running on the upcoming iOS11 only on Apple devices (iPad, IPhone), while this plugin is running on all current Windows / iOS / Android devices.
  3. This plugin can do image recognition / tracking.

Hi,

Can you please explain your problem a little more ? You can use the WidgetInteraction component like in any unreal project.
I would attach the component to the AR camera , and track the camera not the subject. You can enable the debug option for the WidgetInteraction component to see what is happening.

Hi,

I recently purchased your Unreal4AR plugin. I am looking forward to try it out myself, but am having trouble using it on my system.

When I add the plugin to the plugins folder of a new project, I get a message:

ā€œMissing or incompatible modules in ARToolkitPlugin pluginā€

I cannot open the project nor does the plugin work. Can you advice how I can set this up. Probably I am doing something stupid wrong?

Thanks!

Hello,

Did you follow the the Getting Started tutorial ?
http://www.unreal4ar.com/getting-started-full-version/

You need to compile the plugin first for your installed Unreal Engine version .

If you still get this message click on the non disable option. (The 1.9 version is set to ue4.17 by default, you can change that in the uplugin file.)
Works on both 4.16 and 4.17.

Hi guys,

Iā€™m having some issue trying to export the app to google play.
I canā€™t fix the LibPNG vunerability error, even following every tutorial found in internet.
Some things i`ve already tried:
-Deleted the libpng.a in the plugin folder but didinā€™t work.
-Updated the plugin to UE 4.15, did not work too.

Thanks in advance.

Hi

I was having some difficulties getting the demo to build for iOS so I would just like to share my experiences in case anyone else is having difficulties. First a huge thanks to @digitald82! Most of this is derived from his findings, but I just wanted to expand a bit on it. I am brand new to Mac and iOS development so please forgive me if some of these steps are obvious anyone familiar with development on those platforms already! I am developing on macOS Sierra 10.12.6, Unreal 4.17.2, and targeting iOS 9.0 on an iPad Pro.

Expanding on the steps in the getting started guide for the full version:

  1. You will need a Mac computer to package your code project for IOS devices!
  2. Setup your iOS development environment https://docs.unrealengine.com/latestā€¦art/index.html
  3. unreal engine editor (4.17)
  4. Create a new C++ code project without starter content targeting mobile
  5. Copy all files from SampleProject directory to your new project directory and merge the folders
  6. In UE4 editor File -> Refresh xcode project (optional) ā€“ Iā€™m not sure what this doesā€¦
  7. Exit from unreal engine editor
  8. your new project from epic launcher
  9. You should be prompted about ā€œThis project requires the ā€˜ARToolkitPluginā€™ plugin which is not compatible with the current engineā€¦ā€ ā€œWould you like to disable it?ā€, click No.
  10. When it prompts to rebuild the plugins, click Yes.
  11. Go to edit -> project settings -> Platforms -> iOS and import your mobile provision and certificate files.
  12. Also on the iOS settings, update the Bundle Display Name, Bundle Name, and Bundle identifier. If this all matches with your provisioning profile, then the profile and related certificate should turn green. Also as a side note, I havenā€™t had much luck using wild cards so my bundle identifier is specific to my app. You also need to add the additional plist data:

<key>NSCameraUsageDescription</key><string>Camera is used for AR functionality</string>

  1. Project settings -> Project -> Maps & Modes, set your Game Default Map to ARToolkitAdvanced. While youā€™re here, you can set the Editor Startup Map to the same thing if you like.
  2. Project settings -> Project -> Packaging, set Additional Non-Asset Directories To Copy to your projectā€™s Content/ARToolkit folder. Digitald82 had also mentioned the Additional Non-Asset Directories to Package as also being set to the ARToolkit folder, but I didnā€™t find this one to have an effect one way or the other.
  3. File -> Package -> iOS. Select a directory for your iOS build and click Open. The first time you package your project, this takes a very long time if you have a slow machine like mine, but itā€™s much more tolerable after that. :slight_smile: In my case, I made a Package directory in my project.
  4. After the packing is done, we need to check that the case of the file structure aligns with what the plugin is looking for. Go to your Package/IOS directory, make a copy of your .ipa and rename it .zip.
  5. Extract the zip to produce a payload folder. Inside there will be your app. Right click and ā€œShow Package Contentsā€.
  6. In the packaged contents, navigate to cookeddata -> <Your App Name> -> content -> ARToolkit and take note of the case of the subfolders. In my case there are ā€œDataā€ and ā€œDataNFTā€.
  7. Close the Unreal Editor and go to your project directory -> Plugins -> ARToolkitPlugin -> Source -> ARToolkitPlugin -> Private and edit ARToolkitDevice.cpp.
  8. On line 676, check that the case of ā€œDataā€ matches what you found in step 18. In my case, it was a lower case ā€œdataā€.
  9. On line 968, check that that the case of ā€œDataā€ matches what you found in step 18. In my case, it was a lower case ā€œdataā€.
  10. On line 1054, check that that the case of ā€œDataNFTā€ matches what you found in step 18. In my case, it was a lower case ā€œdatanftā€.
  11. Save your file and close.
  12. Navigate to your project folder -> Plugins -> ARToolkitPlugin and trash the Intermediate and Binaries folder. Reload your UE4 project and it will again prompt you about the incompatible plugin (click No), and then rebuild (click Yes).
  13. Assuming that you had to change your plugin source, package your project again (it should be much faster this time).
  14. To transfer your ipa to your device, you need to use Xcode (the ability to do so with iTunes has been removedā€¦). Load up Xcode and go to Window -> Devices and Simulators.
  15. Assuming your device is plugged in, click the + under the installed apps, navigate to your ipa and click open.

Your device should now have the demo loaded and it should prompt you for camera access when you open the app for the first time.

One other bonus note. To get the UE4 logs from your device, while the device is plugged in, from the Devices and Simulators window, select the installed app, then click the gear and select Download Containerā€¦ Select the location for it and after it is copied, right click to Show Package Contents. In the AppData -> Documents -> <app name> -> Saved -> Logs will be your file logs generated from your app. If your still having some troubles, this log might give you some more hints.

Hope this helps anyone new to using this awesome plugin on iOS!

Hi ,

Iā€™ve got a few tracking issues Iā€™ve been trying to figure out.

First, Iā€™ve been playing with different resolutions on the iPad (this iPad is quite capable of running at the highest resolutions at good fps.) Iā€™ve read that the plugin is designed for a 4:3 aspect ratio, but that pretty much negates anything above 640x480 on the iPad. When running on a non-4:3 resolution, the math to dictate the rotations appears to have issues. When looking from the side, the bike looks reasonably grounded on the marker:

But when viewed from the top, the tail of the bike ā€œliftsā€ up:


Could you give me any guidance on where the math can be modified to accommodate a different aspect ratio?

The second issue when running at higher resolutions is that it just canā€™t find the marker. I often find that by backing away (effectively reducing the number of pixels for the marker), it can locate it, and then I can move closer again. The image appears to be clean in the debug view of your example. Can you offer any hints as to what would be affecting the tracking?

The last issue was with the 3x3 matrix markers. I added a Get Marker MATRIX block after the Offset for NFT Marker section and connected a slider output to drive the Marker Id input so the target marker can be changed from the UI. However, regardless of what the input is, the bike always seems to lock onto marker 8. Any thoughts?

Thanks!

Google Play? So you are on android and managed to deal with all the crashes, constrains and bugs? Please share once you publish it. I havenā€™t seen this plugin working well on android and regret spending the money as support was also misleading and evasive.

Hi Ivankio,

You can find an android demo on google play:

What do you mean by misleading and evasive support? Did you write me to info@unreal4ar.com ? I am sorry if I could not help with your issue.

Thanks,

Hi Bruno,

Can you send me your build log ? I havenā€™t seen this error before, nor reported by others. Do you have multiple versions of libPNG.a ?

Thanks,

Hi,

iPads are all 4:3 ? Why do you want to use a non 4:3 resolution ? The plugin wonā€™t work well with other aspect ratios, that is why you have tracking issues.

ā€œThe last issue was with the 3x3 matrix markers. I added a Get Marker MATRIX block after the Offset for NFT Marker section and connected a slider output to drive the Marker Id input so the target marker can be changed from the UI. However, regardless of what the input is, the bike always seems to lock onto marker 8. Any thoughts?ā€

Can you please post a screenshot of your blueprint ?

Thanks,

Hi ,

we had some more detailed discussions offline, but here is the result for anyone having issues running Unreal4AR. As a reminder, I got Unreal4AR working, but the moment I would start the project, the whole thing would crash without clear error message. I could however run it on other PCs, so the problem seemed to be located on one PC.

After a long time, I finally found out what was the problem with this PC.

This PC has installed Windows 10 Enterprise N. The N means that certain packages do not come preinstalled. Apparently, Windows Media Pack does not come preinstalled, which means that for example, a smartphone connected to the PC will not show up in Explorer. This is to enable enterprises to disable access to USB devices.

As a result, the project naturally got into trouble finding the camera. Now that I installed this Windows Media Pack, all works fine!

The learning for me also is that if I have specific issues on any PC, I need to double check it has nothing to do with packages not installed due to the version of Windows.

Thanks again for your help! Itā€™s a great project!

Hi Evarest,

Thank you for the info! I had no prior experience with Windows 10 N.