Augmented Reality for UE4

[AdamHorvath]

Adam how can you set the actual camera in the ARToolkit Advanced Example as a Portrait in android? Actually the project it's set to Portrait but the camera displays in landscape:

Currently you can package the plugin with landscape settings for mobiles. (I'll find a way to make it work in portrait mode)

As a temporary solution:

To use an app in portrait mode, you need to rotate the user interface by 90 degrees, and package the app in landscape settings.

[DavidRR]

https://youtu.be/piukHgnVqcw
Quick vid of AR plugin with root motion animation and physics enabled.

Adam - if you have any tips on how I might enable the default 2D barcodes for use with the plugin, it would be a big help. I'm hoping it will just require setting the marker detection mode and recompiling but searches are failing to locate the init code.

[CornKrue]

Hi Adam,

Great Job with the plugin. We are building a mobile 3D strategy game based on it and it is working impressively:
https://www.youtube.com/watch?v=yzhhIB3_JbQ

[DavidRR]

Crash on resolution change

Hi Adam. Camera resolution change results in a crash in v1.3 - v1.4. I have 'Show Pin' checked on Init. The Pin Properties dialog appears set to MJPG, 640 x 480, 30 FPS. Changing the output size to 960 x 720 or any other size results in the crash below:

Access violation - code c0000005 (first/second chance not available)

""

UE4Editor_ARToolkitPlugin!arLabelingSubDBR3C()
UE4Editor_ARToolkitPlugin!arDetectMarker()
UE4Editor_ARToolkitPlugin!FARToolkitDevice::DetectMarkers() [d:\unreal projects\arpluginv14\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitdevice.cpp:1013]
UE4Editor_ARToolkitPlugin!FARToolkitDevice::UpdateDevice() [d:\unreal projects\arpluginv14\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitdevice.cpp:174]
UE4Editor_Engine!UActorComponent::ConditionalTickComponent() [c:\users\david\documents\github\unrealengine\engine\source\runtime\engine\private\components\actorcomponent.cpp:1139]
UE4Editor_Engine!FActorComponentTickFunction::ExecuteTick() [c:\users\david\documents\github\unrealengine\engine\source\runtime\engine\private\components\actorcomponent.cpp:679]

Running in UE Editor, on Windows 10. Camera is a Logitech HD C615

[AdamHorvath]

Hi Adam. Camera resolution change results in a crash in v1.3 - v1.4. I have 'Show Pin' checked on Init. The Pin Properties dialog appears set to MJPG, 640 x 480, 30 FPS. Changing the output size to 960 x 720 or any other size results in the crash below:

Access violation - code c0000005 (first/second chance not available)

""

UE4Editor_ARToolkitPlugin!arLabelingSubDBR3C()
UE4Editor_ARToolkitPlugin!arDetectMarker()
UE4Editor_ARToolkitPlugin!FARToolkitDevice::DetectMarkers() [d:\unreal projects\arpluginv14\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitdevice.cpp:1013]
UE4Editor_ARToolkitPlugin!FARToolkitDevice::UpdateDevice() [d:\unreal projects\arpluginv14\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitdevice.cpp:174]
UE4Editor_Engine!UActorComponent::ConditionalTickComponent() [c:\users\david\documents\github\unrealengine\engine\source\runtime\engine\private\components\actorcomponent.cpp:1139]
UE4Editor_Engine!FActorComponentTickFunction::ExecuteTick() [c:\users\david\documents\github\unrealengine\engine\source\runtime\engine\private\components\actorcomponent.cpp:679]

Running in UE Editor, on Windows 10. Camera is a Logitech HD C615

Hi David,

I could not reproduce this crash, worked well on my end. Can you please try it with the unmodified demo project ?

In Saved/logs/ there should be a log file.

Can you please open it and locate the following part ?

[2016.02.01-12.50.53:260][572]ARToolkit: Init started
[2016.02.01-12.51.04:037][572]ARToolkit: Camera resolution (960,720)

[2016.02.01-12.51.04:037][572]ARToolkit: Camera pixel format: 1
[2016.02.01-12.51.04:040][572]ARToolkit: Video texture created (960,720)

[AdamHorvath]

https://youtu.be/piukHgnVqcw
Quick vid of AR plugin with root motion animation and physics enabled.

Adam - if you have any tips on how I might enable the default 2D barcodes for use with the plugin, it would be a big help. I'm hoping it will just require setting the marker detection mode and recompiling but searches are failing to locate the init code.

Very nice project!

What software are you running on your tablet ? How does it communicate with UE4 ?

I'll check the 2D barcodes, and let you now if I find out how to make them work.

[AdamHorvath]

Hi Adam,

Great Job with the plugin. We are building a mobile 3D strategy game based on it and it is working impressively:
https://www.youtube.com/watch?v=yzhhIB3_JbQ

Wow, that is great!

You can disable pose filtering for fiducial markers to prevent the drifting effect. http://www.unreal4ar.com/set-filter/

[haonty_pia]

I purchased ARToolkit Plugin.
I don't know how can I connect a Matinee to BLUEPRINT.
I connected the Matinee to BeginPlay, and press the play, during the time that the Matinee is being played come out black.
If it is not connected to the Matinee, it is executed.
I want to know the connection node of Matinee and BluePrint.
It is too difficult for me. Please tell me~~~~~

[Taurustar]

thanks for the answer about camera rotation.... I have another question, how can I change the camera resolution for android devices?

[AdamHorvath]

thanks for the answer about camera rotation.... I have another question, how can I change the camera resolution for android devices?

Default video resolution on iOS is 352x288, and 320x240 on Android. I optimized the plugin for performance, and compatibility to run on low / mid-range devices, also I found that marker detection is more reliable on lower resolutions.

It will be controllable from blueprint in the future plugin versions.

Currently you can only change it from code.

IOS:

ARToolkitDevice.cpp

NSString *vconf = [NSString stringWithFormat : @"%@ %@ %@ %@ %@", @"-format=BGRA", @"-fliph", @"-flipv", @"-preset=cif", @"-position=rear"];

-preset parameter can be::

cif, 480p, 720p, 1080p, low, medium, high


Android:

Change values from 320x240 to 640x480 or higher.

AndroidJNI.cpp line 95

//Camera
FrameWidth = 320;
FrameHeight = 240


GameActivity.java

surfaceTexture.setDefaultBufferSize(320,240);

later on the same file....

cameraParam.setPreviewSize(320, 240);


Best Regards,
Adam

[AdamHorvath]

I purchased ARToolkit Plugin.
I don't know how can I connect a Matinee to BLUEPRINT.
I connected the Matinee to BeginPlay, and press the play, during the time that the Matinee is being played come out black.
If it is not connected to the Matinee, it is executed.
I want to know the connection node of Matinee and BluePrint.
It is too difficult for me. Please tell me~~~~~

You need to expose your blueprint variables to matinee first.

Check out the link below:
https://docs.unrealengine.com/latest...nts/index.html

[DavidRR]

What software are you running on your tablet ? How does it communicate with UE4 ?

Using the Open Sound Control protocol over wifi, monsieurgustav/UE4-OSC plugin, and Lemur on the iPad.

[DavidRR]

Hi David,

I could not reproduce this crash, worked well on my end. Can you please try it with the unmodified demo project ?

In Saved/logs/ there should be a log file.

Can you please open it and locate the following part ?

The unmodified advanced demo from 1.4 is fine using either camera. From my project:

[2016.02.02-22.18.40:485][595]ARToolkit: BisonCam, NB Pro

[2016.02.02-22.18.40:485][595]ARToolkit: Device path: \\?\usb#vid_5986&pid_014c&mi_00#7&a9de3d&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global

[2016.02.02-22.18.40:485][595]ARToolkit: Logitech HD Webcam C615

[2016.02.02-22.18.40:485][595]ARToolkit: Device path: \\?\usb#vid_046d&pid_082c&mi_02#6&88cf4eb&0&0002#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\{bbefb6c7-2fc4-4139-bb8b-a58bba724083}

[2016.02.02-22.18.40:485][595]ARToolkit: Number of webcams: 2
[2016.02.02-22.18.40:485][595]ARToolkit: Init started
[2016.02.02-22.18.47:202][595]ARToolkit: Camera resolution (960,720)

[2016.02.02-22.18.47:202][595]ARToolkit: Camera pixel format: 1
[2016.02.02-22.18.47:205][595]ARToolkit: Video texture created (960,720)

[2016.02.02-22.18.47:205][595]ARToolkit: Param load start
[2016.02.02-22.18.47:205][595]ARToolkit: Param load success
[2016.02.02-22.18.47:205][595]ARToolkit: Camera init succesfull!
[2016.02.02-22.18.47:308][595]ARToolkit: AR Patt Handle created

[2016.02.02-22.18.47:308][595]ARToolkit: 3D Handle created

[2016.02.02-22.18.47:308][595]ARToolkit: ARToolkit init succesfull!
[2016.02.02-22.18.47:308][595]ARToolkit: NFT init started...
[2016.02.02-22.18.47:308][595]ARToolkit: Using NFT tracking settings for more than one CPU.

[2016.02.02-22.18.47:308][595]ARToolkit: NFT init successfull...
[2016.02.02-22.18.47:474][595]ARToolkit: Init complete !!

[2016.02.02-22.18.47:475][595]LogBlueprintUserMessages: [ARToolkitBaseAdvanced_C_1] 960x720
[2016.02.02-22.18.47:475][595]ARToolkit: Reading C:/Users/David/Documents/Perforce/ChoreoAR/Content/ARToolkit/Data/patt.Hiro
[2016.02.02-22.18.47:475][595]ARToolkit: Hiro loaded

[2016.02.02-22.18.47:475][595]ARToolkit: Reading C:/Users/David/Documents/Perforce/ChoreoAR/Content/ARToolkit/Data/patt.Kanji
[2016.02.02-22.18.47:475][595]ARToolkit: Kanji loaded

[2016.02.02-22.18.47:475][595]ARToolkit: Reading C:/Users/David/Documents/Perforce/ChoreoAR/Content/ARToolkit/Data/patt.sample1
[2016.02.02-22.18.47:476][595]ARToolkit: sample1 loaded

[2016.02.02-22.18.47:476][595]ARToolkit: Reading C:/Users/David/Documents/Perforce/ChoreoAR/Content/ARToolkit/Data/patt.sample2
[2016.02.02-22.18.47:476][595]ARToolkit: sample2 loaded

[2016.02.02-22.18.47:476][595]PIE: Info Play in editor start time for /Game/Maps/UEDPIE_0_ARToolkitAdvanced 6.394
[2016.02.02-22.18.47:477][595]LogBlueprintUserMessages: Late PlayInEditor Detection: Level '/Game/Maps/ARToolkitAdvanced.ARToolkitAdvanced:PersistentLevel' has LevelScriptBlueprint '/Game/Maps/ARToolkitAdvanced.ARToolkitAdvanced:PersistentLevel.ARToolkitAdvanced' with GeneratedClass '/Game/Maps/ARToolkitAdvanced.ARToolkitAdvanced_C' with ClassGeneratedBy '/Game/Maps/ARToolkitAdvanced.ARToolkitAdvanced:PersistentLevel.ARToolkitAdvanced'
[2016.02.02-22.18.47:802][595]LogCrashTracker: 


[2016.02.02-22.18.55:931][595]LogCrashTracker: 


[2016.02.02-22.18.55:931][595]LogWindows:Error: === Critical error: ===
Fatal error!

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0x8a8d1004

UE4Editor-ARToolkitPlugin.dll!arLabelingSubDBR3C()
UE4Editor-ARToolkitPlugin.dll!arDetectMarker()
UE4Editor-ARToolkitPlugin.dll!FARToolkitDevice::DetectMarkers() [d:\unreal projects\arpluginv14\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitdevice.cpp:1013]
UE4Editor-ARToolkitPlugin.dll!FARToolkitDevice::UpdateDevice() [d:\unreal projects\arpluginv14\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitdevice.cpp:174]
UE4Editor-Engine.dll!UActorComponent::ConditionalTickComponent() [c:\users\david\documents\github\unrealengine\engine\source\runtime\engine\private\components\actorcomponent.cpp:1139]
UE4Editor-Engine.dll!FActorComponentTickFunction::ExecuteTick() [c:\users\david\documents\github\unrealengine\engine\source\runtime\engine\private\components\actorcomponent.cpp:679]
UE4Editor-Engine.dll!FTickFunctionTask::DoTask() [c:\users\david\documents\github\unrealengine\engine\source\runtime\engine\private\ticktaskmanager.cpp:141]

I have removed the NFT marker load loop and don't Get Marker for NFT.

I've been unable to recreate the crash in the demo by making my changes but will keep investigating.

[DavidRR]

Adam,

Will you further explain the Primary output on GetMarker? How are specific markers configured as primary? Maybe it will help to let me quickly explain what would be useful. I'd like to use 1 or up to 5 unique markers for camera control - one marker for each corner of the playfield. Ideally at least 2 markers will be visible to the webcam at all times for more accurate camera placement. Other markers will be for game assets / characters and should not influence the camera.

Depending on what happens with 2D barcodes the technique I'm considering could change. Currently I should be able to iterate thru two groups of markers, checking for visibility and position. One group would be the camera control group and the second group would be for game assets. If the marker types can be configured as primary when loaded then it would be very useful for new nodes to be created that would return an array of visible primary markers and another for non-primary markers. Hopefully the returned array would include the position and rotation of each marker so that additional queries aren't needed.

Best Regards,
David

[DavidRR]

Hi again Adam,

Are you aware that one of the cube planes in the demo is not fully transparent? I have no skills in materials and haven't been able to make any useful changes while keeping the object shadow. This edge will move with the camera. Efforts to post a photo here are failing with database error but the problem is pretty easy to spot if you move the camera around and keep it low to look off into the horizon.

David