Unreal, lets make some noise for AR support.

General Feedback & Requests
1

Dear Unreal staff and community,

let’s start with the good things:
During the last six month I mainly focused on creating AR-prototypes for some big clients. I did this mainly to check if there is a real market for AR apps. The results were absolutely positive, the interest in this area is huge. Even clients who initially showed little interest (“Yeah, yeah, we know AR already. We heard and read about it…”) changed their minds after they experienced it. Their scepticism changed into enthusiasm.

Now the sad part:
There is currently no production-ready solution for creating Augmented Reality apps in Unreal. The two major players in the field of AR apps, metaio and Vuforia, both provide various SDK’s and even a Unity plugin - but no Unreal Engine support.

Several users, besides myself, have posted dozens of requests on both forum sites, e.g.:
Vuforia:
https://developer.vuforia.com/forum/general-discussion/ue4-integration
https://developer.vuforia.com/forum/general-discussion/wish-list (you have to click through the pages to find the upvotes on this long wishlist)

metaio:
http://helpdesk.metaio.com/questions/31856/unreal-4-support

Unfortunately, the posted requests from the community for Unreal Engine support did not seem to have a big impact. There are still, as far as I know, no plans to support UE.

I think we should change that :slight_smile:

So I would like to ask Unreal, if they, as a company, could reach out to Vuforia and/or metaio and help pointing out that there is a community that is very interested in using their tools to develop great AR apps in Unreal Engine. Maybe Unreal could even offer some technical support developing the plugins? As mentioned above, Vuforia and metaio both already provide various SDK’s (C++, Java, Objective-C etc.), so maybe a plugin could be developed with relatively little effort (just a guess, I’m not a C++ programmer).

In the meantime, I’ll keep posting requests on the metaio- and Vuforia-forums.
If you are interested in developing AR in Unreal, it would be great if you do so too :slight_smile:
metaio forum: http://helpdesk.metaio.com/
vuforia forum: https://developer.vuforia.com/forum

Cheers,
Vector3

2

Problem is UE4 does not support alightment with envriament which is needed for such a thing, if thats done it can support any AR technology, atleast it might be easy to implement any by anyone. Implementing whole support for that might be too big to do right now, but Microsofts hololens and other AR VRs might come might motivate Epic to do so, camera AR would be first natural step

3

Hi,

just to be sure that there is no misunderstanding here:
the idea is **not ** that Unreal implements its own AR solution - that would indeed be quite a huge task.
And it would actually be a reinvention of the wheel as both frameworks mentioned above already do most of the hard work (marker recognition and tracking from camera feed, calculation of camera matrices etc.).

So the actual task for now would be the creation of plugins for Unreal Engine that enable us to access the tracking data produced by these two frameworks. As I said, both companies already offer plugins for Unity (and various SDK’s) - so why not have plugins for Unreal Engine as well?

It would be great if Unreal could help pushing the development of such plugins.

In the end I would love to see UE established as a serious platform for AR development - ranging from “classical” marker based AR to upcoming AR-technologies like Microsofts Hololens.

Unreal, are you in on it…? :slight_smile:

4

Yes, i know what you mean, what i’m saying is there currently no support for any AR technology, UE4 as i know does not even support camera support to begin with (To display it as background), which would be also useful to other things. If Epic gonna implement AR to UE4, i think best would be is to create common interface for all those technologies and then attach those frameworks that you talking about, this way whole thing will be more portable, because you need to keep in mind there other platforms and AR technologies which could benefit from that.

5

Hm…
Sure, it would be nice to have a common interface for all possible AR (and VR) technologies. But wouldn’t that require that all these technologies are build upon some unified standard API as well? I honestly doubt that we will see such a standardized API across the industry in the near future.

So what would be the next steps?
Could Unreal ask Vuforia and metaio if they are willing to develop the plugins - together with Unreal if necessary (e.g. device camera support etc.)?
If its possible for Unity, it should be possible for UE as well, shouldn’t it? :slight_smile:

I would also be really interested in the opinion of some Unreal devs here. Is this task really completely illusory?

Cheers.

6

Thats what common interface exacly is for, how do you think UE4 works on some many platforms? :stuck_out_tongue: Because different APIs are wrapped with common one down to level of C++ core liberies (thats why you don’t use standard C++ even if you can but only UE4 APIs). UE4 unifies all online services (Steam, Xbox Live, PSN etc) under common interface (onlinesubsystem) as well as render liberies (DirectX, OpenGL etc in to RHI), and i think VRs to in 4.8 (not exacly sure).

What is common in for AR technology which UE4 game code will need, first camera support which UE4 missing (which is need to be supported by platfrom interface), the camera positioning and tracking state (if glyth i lost or not). Epic can create common interface for such things so they can be used in any platfrom and any technology, it will just need module or plugin that will operate those APIs, which will be brige between UE4 common API and specific technology plugin.

I think it will be lot nicer then making those companies implement AR in different incompatible ways, it would be a mess

7

Well, I thought that’s the whole point of the plugin system (and the open-source mentality): to be able to extend the engine with functionalities it doesn’t natively provide in its current state.
If Epic would have to implement a common interface for every new functionality a third party plugin provides, then this would be a serious design failure - in my eyes.

Your argument may be true if AR becomes a build-in core platform feature. Then you need an API third-party developers can build against.
Until this happens, we have to rely on plugins built without a common interface - i can’t see why this should be a bad thing.

8

Problem is Qualcomm will only work on Android and iOS, in order to port it to other platform you would need use something else which won’t be compatible, UE4 is designed in the way that regardless what you code it will work elsewhere, thats why i think it would be better for Epic to implement it then making 3rd parties do it. Also reason why you don’t see those companies interested in making plugin to UE4 because 1. Unity dominates on mobile market and it will for longer time because Unity doing a lot better in mobile 2. There no distribution platfrom for plug-ins as plug-in system it self is not 100% ready

9

Would be a keen supporter of UE AR if epic could include some code…

10

Here we are, 5 months after my initial post. A good time to ask for a status update, I guess :):

Has there been made any progress in regards of implementing AR functionality in Unreal Engine?
Can we expect Unreal Engine to support Augmented Reality in the near future (i.e. early 2016)?

As mentioned in my previous posts, I’m not talking about Hololens here in the first place (this device might be consumer-ready in 3 years at the earliest), but the “classical” Tablet/Smartphone based approach, which is production-ready right now - I actually just finished developing 2 AR-apps for medical companies. I couldn’t use Unreal Engine of course.

I appreciate that Epic goes all-in for VR, but leaving AR behind might be a big mistake. I honestly don’t understand, why Epic is leaving this whole field to Unity. It seems, I’m not the only one who feels this way (e.g. https://forums.unrealengine.com/showthread.php?57795-Request-for-AR-support-to-roadmap).

If working togther with Vuforia to bring AR to Unreal is not an option, there is now even an open-source SDK out there (http://artoolkit.org/)…

So again, Epic, when will Unreal Engine be AR-ready…?

Cheers.

11

Checkout my new thread in the work in progress section about ARToolkit integration.

12

as long i know ARToolkit is a opencv implementation working with “markers” images and is very cool, real AR is archive with a lot of image processing algorithm, some aproximation can be done with opencv, in order to track a real image like whatever picture taken from whatever device!, you must use feature detector, then create image descriptors and from this, you can track a image in a device and founds the homography and comparising with the original image pattern, finds homography allow us know the perspective transform for the camera!, undestands that kind of algorithm is not easy task, image processing is really overhelmet if you dont know maths!, as long i know again!, qualcomm offers for free use, his library called FastCV exclusive for mobile performance, and is very COOL, i used some time ago for face track, vuforia was develop with FastCV, i dont know about metaio.

the reason for not good results in UE right now is her texture buffer, in UE we have some copies from the image buffer for this, 1 in cpu and 1 for gpu, thats why all user cant play video texture bigger that 720p, UE thinks solve this in a future release, (i read in another post, maybe for 4.10, not sure), if this solves!, so the device buffer maybe pass directly in the texture without any copy, which is cool we can now play videotextures much biggers!, anywhy this is not a big deal for desktop, but for mobile is another history!
i really wait for this!, and when we have something, i will expect good results from comunity users like AdamHorvath! is doing very great work!

13

I have used unity 5 with vuforia in MY AR developments, and it’s reaally painful to get into unity knowing unreal, luckily you can setup a template to show the vuforia work, but triying to get from/to ue4 to unity is painful at best.