Augmented Reality for UE4

[expose]

Hi Adam and everyone!

I wonder if this plugin could support the video capture card? Or is there any other solution for my project that the video stream from a remote quadcopter have to be transmitted to unreal engine as texture? Now I use a video capture card to receive the video and get the image using a C++ API. But I have no idea how can I transfer the pixel buffer variable into the texture that could be used in unreal engine.
Anyone know the answer?

hi there. i have a decklink card in my computer. and the plugin sees it (try several dev num to find it, an tick the "show pin" to identify it) didn't try it with an input for now, but it seems that every hardware that windows sees as video input can be used with the plugin. i'll test the decklink this week and tell you

[Totomoon]

Thank you ~I have tested that and it works [MENTION=412498]expose[/MENTION]

[Totomoon]

Hi@AdamHorvath ， I bought a personal version. I want to integrate your plugin with my existing project. Can your plugin works as a normal plugin that I only add it to the plugin folder,then I can use the blueprint node in blueprint graph?
I think I can't do as your guide because it needs me copy the entire contents to my existing projects. I think it will influence my existing project. Right?
How can I make it work?
When I use your plugin in my experimental blueprint, and play it. The editor will crash. And in the error log, shows as following, everything stops at init started.. But the free sample project works well.
my experimental blueprint is like this:
[2017.02.14-08.20.15:025][124]LogPlayLevel: Creating play world package: /Game/FirstPersonBP/Maps/UEDPIE_0_FirstPersonExampleMap
[2017.02.14-08.20.15:055][124]LogPlayLevel: PIE: StaticDuplicateObject took: (0.029659s)
[2017.02.14-08.20.15:056][124]LogAIModule: Creating AISystem for world FirstPersonExampleMap
[2017.02.14-08.20.15:056][124]LogPlayLevel: PIE: World Init took: (0.001411s)
[2017.02.14-08.20.15:057][124]LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/FirstPersonBP/Maps/FirstPersonExampleMap.FirstPersonExampleMap to /Game/FirstPersonBP/Maps/UEDPIE_0_FirstPersonExampleMap.FirstPersonExampleMap (0.031567s)
[2017.02.14-08.20.15:081][124]LogInit: XAudio2 using 'Realtek Digital Output (Realtek High Definition Audio)' : 2 channels at 48 kHz using 16 bits per sample (channel mask 0x3)
[2017.02.14-08.20.15:091][124]LogInit: FAudioDevice initialized.
[2017.02.14-08.20.15:127][124]LogLoad: Game class is 'FirstPersonGameMode_C'
[2017.02.14-08.20.15:130][124]LogWorld: Bringing World /Game/FirstPersonBP/Maps/UEDPIE_0_FirstPersonExampleMap.FirstPersonExampleMap up for play (max tick rate 0) at 2017.02.14-16.20.15
[2017.02.14-08.20.15:130][124]LogWorld: Bringing up level for play took: 0.001892
[2017.02.14-08.20.15:132][124]ARToolkit: Android orientation: Landscape
[2017.02.14-08.20.15:165][124]ARToolkit: USB2.0 PC CAMERA

[2017.02.14-08.20.15:167][124]ARToolkit: Device path: \\?\usb#vid_18ec&pid_5850&mi_00#6&3363d1ac&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global

[2017.02.14-08.20.15:167][124]ARToolkit: Number of webcams: 1
[2017.02.14-08.20.15:167][124]ARToolkit: Init started

Then if I restarted the engine, the information log will say:
error /Game/FirstPersonBP/Blueprints/FirstPersonCharacter : cann't find file of resource。 /Script/ARToolkitPlugin
load error: /Script/ARToolkitPlugin.ARToolkitComponent quoter: K2Node_VariableGet_1479
load error /Script/ARToolkitPlugin.EArPatternDetectionMode quoter K2Node_CallFunction_128
load error /Script/ARToolkitPlugin.EDeviceOrientation quoter K2Node_CallFunction_128

[AdamHorvath]

Hi,

You can find a tutorial on the link below to setup everything from scratch.
http://safefromrobots.com/?p=149

BR,
Adam

Hi@AdamHorvath ， I bought a personal version. I want to integrate your plugin with my existing project. Can your plugin works as a normal plugin that I only add it to the plugin folder,then I can use the blueprint node in blueprint graph?
I think I can't do as your guide because it needs me copy the entire contents to my existing projects. I think it will influence my existing project. Right?
How can I make it work?
When I use your plugin in my blueprint, and play it. The editor will crash. And in the error log, shows as following, everything stops at init started.. But the free sample project works well.
[2017.02.14-08.20.15:025][124]LogPlayLevel: Creating play world package: /Game/FirstPersonBP/Maps/UEDPIE_0_FirstPersonExampleMap
[2017.02.14-08.20.15:055][124]LogPlayLevel: PIE: StaticDuplicateObject took: (0.029659s)
[2017.02.14-08.20.15:056][124]LogAIModule: Creating AISystem for world FirstPersonExampleMap
[2017.02.14-08.20.15:056][124]LogPlayLevel: PIE: World Init took: (0.001411s)
[2017.02.14-08.20.15:057][124]LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/FirstPersonBP/Maps/FirstPersonExampleMap.FirstPersonExampleMap to /Game/FirstPersonBP/Maps/UEDPIE_0_FirstPersonExampleMap.FirstPersonExampleMap (0.031567s)
[2017.02.14-08.20.15:081][124]LogInit: XAudio2 using 'Realtek Digital Output (Realtek High Definition Audio)' : 2 channels at 48 kHz using 16 bits per sample (channel mask 0x3)
[2017.02.14-08.20.15:091][124]LogInit: FAudioDevice initialized.
[2017.02.14-08.20.15:127][124]LogLoad: Game class is 'FirstPersonGameMode_C'
[2017.02.14-08.20.15:130][124]LogWorld: Bringing World /Game/FirstPersonBP/Maps/UEDPIE_0_FirstPersonExampleMap.FirstPersonExampleMap up for play (max tick rate 0) at 2017.02.14-16.20.15
[2017.02.14-08.20.15:130][124]LogWorld: Bringing up level for play took: 0.001892
[2017.02.14-08.20.15:132][124]ARToolkit: Android orientation: Landscape
[2017.02.14-08.20.15:165][124]ARToolkit: USB2.0 PC CAMERA

[2017.02.14-08.20.15:167][124]ARToolkit: Device path: \\?\usb#vid_18ec&pid_5850&mi_00#6&3363d1ac&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global

[2017.02.14-08.20.15:167][124]ARToolkit: Number of webcams: 1
[2017.02.14-08.20.15:167][124]ARToolkit: Init started

[Totomoon]

Hi,

You can find a tutorial on the link below to setup everything from scratch.
http://safefromrobots.com/?p=149

BR,
Adam

Thanks for your timely help!
I have build the plugin successfully and the intermediate folder come up in the plugin folder. But when I put it in the plugins folder of the engine, why the editor plugins does not contain it? It should contain all the plugins put under the /plugin folder, right?

[Totomoon]

Also, the most urgent thing is that the editor crash in the init started step after I've build the plugin!

[Totomoon]

Hi,

You can find a tutorial on the link below to setup everything from scratch.
http://safefromrobots.com/?p=149

BR,
Adam

Also, the most urgent thing is that the editor crash in the init started step after I've build the plugin!

[eden_kee]

I have found a solution for my previous post

I was using the wrong condition for my branch action

then, to compensate the pause function error, i set the play rate to '0'

[Totomoon]

Editor continue crashes!

[MENTION=35119]AdamHorvath[/MENTION], I don't know why but this time it could complete init but still crashes!!!
[2017.02.14-10.30.23:940][568]LogLoad: Game class is 'GameModeBase'
[2017.02.14-10.30.23:944][568]LogWorld: Bringing World /Temp/UEDPIE_0_Untitled_1.Untitled_1 up for play (max tick rate 0) at 2017.02.14-18.30.23
[2017.02.14-10.30.23:945][568]LogWorld: Bringing up level for play took: 0.003563
[2017.02.14-10.30.23:945][568]ARToolkit: Android orientation: Landscape
[2017.02.14-10.30.23:946][568]ARToolkit: USB2.0 PC CAMERA

[2017.02.14-10.30.23:946][568]ARToolkit: Device path: \\?\usb#vid_18ec&pid_5850&mi_00#6&3363d1ac&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global

[2017.02.14-10.30.23:946][568]ARToolkit: Number of webcams: 1
[2017.02.14-10.30.23:946][568]ARToolkit: Init started
[2017.02.14-10.30.27:817][568]ARToolkit: Camera resolution (640,480)

[2017.02.14-10.30.27:817][568]ARToolkit: Camera pixel format: 1
[2017.02.14-10.30.27:817][568]ARToolkit: Video texture created (640,480)

[2017.02.14-10.30.27:817][568]ARToolkit: Param load start
[2017.02.14-10.30.27:817][568]ARToolkit: Param load success
[2017.02.14-10.30.27:817][568]ARToolkit: Camera init succesfull!
[2017.02.14-10.30.27:858][568]ARToolkit: AR Patt Handle created

[2017.02.14-10.30.27:858][568]ARToolkit: 3D Handle created

[2017.02.14-10.30.27:858][568]ARToolkit: Detection mode : TEMPLATE
[2017.02.14-10.30.27:858][568]ARToolkit: ARToolkit init succesfull!
[2017.02.14-10.30.27:858][568]ARToolkit: NFT init started...
[2017.02.14-10.30.27:858][568]ARToolkit: Using NFT tracking settings for more than one CPU.

[2017.02.14-10.30.27:858][568]ARToolkit: NFT init successfull...
[2017.02.14-10.30.27:886][568]ARToolkit: Init complete !!

[2017.02.14-10.30.28:130][568]LogCrashTracker:
UE4Editor_ARToolkitPlugin!__delayLoadHelper2() [f:\dd\vctools\delayimp\delayhlp.cpp:323]
UE4Editor_ARToolkitPlugin!_tailMerge_ARvideo_dll()
UE4Editor_ARToolkitPlugin!FARToolkitDevice::Init() [c:\users\totomoon\documents\unreal projects\test8\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitdevice.cpp:539]
UE4Editor_ARToolkitPlugin!UARToolkitComponent::Init() [c:\users\totomoon\documents\unreal projects\test8\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitcomponent.cpp:169]
UE4Editor_ARToolkitPlugin!UARToolkitComponent::execInit() [c:\users\totomoon\documents\unreal projects\test8\plugins\artoolkitplugin\source\artoolkitplugin\public\artoolkitcomponent.h:14]
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine

[K-lab srls]

Hi Adam,
can I use this plugin as a qr code reader?
I need to scan some qr code to open the relative level, and I see it can be possible.
How can I open and close the (web) camera every time?
The project is made with UE 4.14 for mobile, Android and iOS (even if I need a Mac..)

Thanks for all

[AdamHorvath]

Hi,

You can only scan ARToolkit compatible markers, it's not compatible with QR codes.

Hi Adam,
can I use this plugin as a qr code reader?
I need to scan some qr code to open the relative level, and I see it can be possible.
How can I open and close the (web) camera every time?
The project is made with UE 4.14 for mobile, Android and iOS (even if I need a Mac..)

Thanks for all

[AdamHorvath]

Is it your custom project or the sample levels included with the plugin ?

[MENTION=35119]AdamHorvath[/MENTION], I don't know why but this time it could complete init but still crashes!!!
[2017.02.14-10.30.23:940][568]LogLoad: Game class is 'GameModeBase'
[2017.02.14-10.30.23:944][568]LogWorld: Bringing World /Temp/UEDPIE_0_Untitled_1.Untitled_1 up for play (max tick rate 0) at 2017.02.14-18.30.23
[2017.02.14-10.30.23:945][568]LogWorld: Bringing up level for play took: 0.003563
[2017.02.14-10.30.23:945][568]ARToolkit: Android orientation: Landscape
[2017.02.14-10.30.23:946][568]ARToolkit: USB2.0 PC CAMERA

[2017.02.14-10.30.23:946][568]ARToolkit: Device path: \\?\usb#vid_18ec&pid_5850&mi_00#6&3363d1ac&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global

[2017.02.14-10.30.23:946][568]ARToolkit: Number of webcams: 1
[2017.02.14-10.30.23:946][568]ARToolkit: Init started
[2017.02.14-10.30.27:817][568]ARToolkit: Camera resolution (640,480)

[2017.02.14-10.30.27:817][568]ARToolkit: Camera pixel format: 1
[2017.02.14-10.30.27:817][568]ARToolkit: Video texture created (640,480)

[2017.02.14-10.30.27:817][568]ARToolkit: Param load start
[2017.02.14-10.30.27:817][568]ARToolkit: Param load success
[2017.02.14-10.30.27:817][568]ARToolkit: Camera init succesfull!
[2017.02.14-10.30.27:858][568]ARToolkit: AR Patt Handle created

[2017.02.14-10.30.27:858][568]ARToolkit: 3D Handle created

[2017.02.14-10.30.27:858][568]ARToolkit: Detection mode : TEMPLATE
[2017.02.14-10.30.27:858][568]ARToolkit: ARToolkit init succesfull!
[2017.02.14-10.30.27:858][568]ARToolkit: NFT init started...
[2017.02.14-10.30.27:858][568]ARToolkit: Using NFT tracking settings for more than one CPU.

[2017.02.14-10.30.27:858][568]ARToolkit: NFT init successfull...
[2017.02.14-10.30.27:886][568]ARToolkit: Init complete !!

[2017.02.14-10.30.28:130][568]LogCrashTracker:
UE4Editor_ARToolkitPlugin!__delayLoadHelper2() [f:\dd\vctools\delayimp\delayhlp.cpp:323]
UE4Editor_ARToolkitPlugin!_tailMerge_ARvideo_dll()
UE4Editor_ARToolkitPlugin!FARToolkitDevice::Init() [c:\users\totomoon\documents\unreal projects\test8\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitdevice.cpp:539]
UE4Editor_ARToolkitPlugin!UARToolkitComponent::Init() [c:\users\totomoon\documents\unreal projects\test8\plugins\artoolkitplugin\source\artoolkitplugin\private\artoolkitcomponent.cpp:169]
UE4Editor_ARToolkitPlugin!UARToolkitComponent::execInit() [c:\users\totomoon\documents\unreal projects\test8\plugins\artoolkitplugin\source\artoolkitplugin\public\artoolkitcomponent.h:14]
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_CoreUObject
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine
UE4Editor_Engine

[motorsep]

[MENTION=35119]AdamHorvath[/MENTION]

I started poking at AR (conceptual stage for now) before any reasonably priced AR HMD solution is out on the market and I am wondering if it's possible to do object tracking with your plugin and UE4. Something similar to what Blender has been doing since 2011:

https://vimeo.com/33283857

Also wondering if you have any near future plans to add support for Project Tango.

[Denis]

Hi Adam, hi everyone !

How do I make my 3D Model stay at his last known position when the marker is lost, and start tracking again when the marker comes back into view?
I've followed these steps :

1. Open the level blueprint and locate the "set hidden in game node"
2. Disconnect the new hidden pin and set it to false
3. It won't disappear if the marker is not visible.

But the 3d Model seems to get back to 0,0,0 and not the last know position when it last had seen the tracker.

How can I fix that?

Thanks in advance,

Denis

[K-lab srls]

Hi,

You can only scan ARToolkit compatible markers, it's not compatible with QR codes.

Hi Adam,
I think I can use a simple QR code inner the black frame, it's correct?
//Anyway how can I open and close the camera every time?
Edit: I try the ARSample and I'm able to open and close camera and the marker recognition as I want!

Thanks for your reply