Earlier this year I was able to attend E3 and as everyone would expect I got to see lots of awesome things that will be coming out in 2016. One of those things is a pair of wireless VR gloves being developed by a company called Manus Machina (http://www.manusmachina.com) The demo I played that uses the gloves was made using Unity and the gloves worked by using the positional tracking system on a Samsung Gear VR. You could think of them similar to how Leap Motion works but the cool part is since the gloves have built in sensors to measure the movement of individual fingers and the palm of each hand you don’t get all the weird stuff that could happen by using IR like Leap Motion has (the Manus E3 demo can be viewed on the homepage of their website).
After playing the demo I explained to them I’m a video game developer who switched from using Unity to Unreal Engine 4 and asked if they had anyone working on other demos for them. They have a few but everyone they are working with is using Unity while one other is using UE4. Without explaining the whole story of how it got to this point I’m now partnered up with these guys to make them a demo. The idea would be to make something that actually shows off hand motions so my goal is to create an interactive demo that allows you to use the gloves to do simple things like push buttons, pull levers, etc. (how many more times will I say “demo”? haha)
An issue came about though during discussions because it seems they do not have a plugin for the gloves to work in UE4 and they had just partnered with Razer so they would like future demos to also work with Razer’s OSVR (Razer 雷蛇中文官方网站). This poses a problem for me because I’m not too skilled in C++ and the end goal is for the Manus gloves to communicate with an OSVR using its positional tracking system to control a ‘Controller Actor’ or rigged character in game. I know some C++ but not enough to be able to develop a plugin for those two devices to work together in UE4.
I really believe this new glove technology is another step for UE4 to grow and I would love to see that happen but unfortunately I’m not skilled enough to make that happen. The things I’ve seen people do from this community is inspiring; Rama makes plugins to give UE4 so much more base functionality it’s a surprise he/she is not a UE4 engine developer, Grizly32 developed the plugin that is used by anyone who is working on Arduino based UE4 projects, ‘getnamo’ made our event driven plugin for Razer’s Hydra motion controller and there is so many more I haven’t named or even heard of. I’m hoping this thread can generate some interest in the Manus gloves so someone from our awesome community can develop the plugin that hopefully will lead to UE4 making VR games even more immersive.
**
I’m not looking to retain rights to this plugin but at the same time I’m even willing to pay someone for this to get done** so we can share it with the community. Like I said, I firmly believe this will benefit UE4 /the community and I would want everyone to have the chance to use them without having to go through the process I’m trying to get through now. Below are all the links you would ever need and if you have any questions just let me know here or PM.
I think the biggest problem that people attempting to use “alternative inputs” is that to my knowledge, UE4 does not support DirectInput, and while there is a request for it to be implemented, it is currently “out of scope with the direction of the engine”.
edit:
Looking at their gloves, it looks like they had the same conclusions as I did, to use flex sensors in the fingers and an IMU near the backhand, but in their case, they used it near the wrist. Not sure why their module is so big (prototype I hope), the IMU/Wireless box should be as thin, if not thinner, than a smartphone (and much, much lighter ofc).
A bit ago Heiko Fink ( aka Cyxx ) developed a fantastic plugin for UE4 for the Perception Neuron mocap suit, using the data from their software and translating them to UE4 using C++ and a custom blueprint, so I think a similar method can be done.
He did that for himself, but then he saw that the community was interested, and also because the developer themself were very late on their UE4 plugin, he released for free ( with optional donations ) and Epic gave him 12k of Unreal Grants for his contribution
What you need to do is to request/download/ask to use their software for the input capture and then, by asking directly on this forum, or to the Manus developers, the manus community or just people who knows how this thing works, ask for help on developing this system, then share the plugin…
I strongly suggest you to start a crowfunding initiative for someone who could develop that, and I think that many will be interested, since the technology is pretty cool
I’m not sure if direct input is required here. As I explained the gloves are to be used with the OSVR’s positional tracking system which I forgot to mention that is done by a IR Camera that comes with the OSVR. I’m not a hardware god or anything so I’m really not sure if what I’m saying is correct but to me it seems direct input would be for stuff like the flight stick that was mentioned in the thread you posted since its not being used in conjunction with anything like a VR headset’s positional tracking. That’s how I’m taking everything though. Please let me know if that’s not the case though since I’m trying to learn as this goes
To answer one of your questions though, Manus is still in development and that won’t be the final product. The versions being shipped out this month are developer kits and the consumer version will be released later in 2016. They are still trying to get it more refined as well as get more HMD’s such as the Oculus Rift to work with them.
The Manus gloves are open source which their SDK can be found here as posted on the first topic post:
I just found the plugin you were talking about made by Cyxx for the Perception Neuron Mocap Suit. He did a lot of work there and that’s inspiring to me, just another person to add to the list of great community members, haha. To be honest I don’t care about getting a grant or anything because my whole goal with game design is to do what everyone should be doing which is pushing the limits of all software and hardware that is available. The Manus gloves might be the next step in the process to getting a full interactive VR environment which is why I really think UE4 and the community can help make that a goal that can be achieved. I would love to see a bunch of tech combined to make the ultimate experience which is what I’m trying to do. Someone that could make a game or something that combines an HMD (Oculus, OSVR, whatever) with the Manus gloves and Virtuix Omni or whatever tech they wanted then we’ll have the start of some amazing VR games for E-Sports, etc.
Could you explain your crowdfunding idea though? I’m interested to know how that would help unless your saying raise money for the Manus UE4 development which would gain attention from other developers. Someone could get interested during the crowdfunding phase and whatever money is raised could go to that developer?
DirectInput probably isn’t a requirement, however, it would make it far easier to implement alternative input devices without a ton of engine modification. I figured it was prototype/development hardware As far as to getting HMD companies to work with them, I would suggest to them to instead, try to get game engine makers to work with them.
Basically, it’s like asking monitor makers to work with keyboard makers. No reason to.
PS: If you have direct access to them, let them know that awhile back, there was a research paper that came out about (I think it was MIT), discovered/created a composite material that increased/decreased elasticity depending on current applied to it. When I saw that, I thought to myself that it would make for a GREAT haptic feedback material in VR.
I see what you mean about direct input, since its not an option though then we’ll have to find another way which like explained the positional tracking from a HMD like the OSVR previously mentioned (when there is a will, there’s a way! haha)
The part I disagree with though is me just reaching out to the HMD developer. The point of seeing if anyone in the UE4 community would like to help is this is a new technology that not even the HMD knows how to use. Who better to make these gloves work with UE4 than the people who develop with UE4 themselves which would be the community (other than the glove developers themselves BUT always nice to work together to accomplish something).
I do have direct contact with ManusMachina since I’m developing a demo for them (which it will be 100% done once the gloves have UE4 support. If I was a hardware guy I might be able to do it but I’m not so I’m constantly scratching my head trying to hook everything together lol).
Manus Machina shipped a pair of gloves to me last week so as soon as I get them I’ll be keeping this thread updated with videos and such on how they work
I am working with 5DT Data Gloves 14 Ultra and want it to work with Unreal Engine 4. I have C++ SDK for 5DT Data Gloves 14 Ultra and further can you guide me for the development of plugin.
(which it will be 100% done once the gloves have UE4 support. If I was a hardware guy I might be able to do it but I’m not so I’m constantly scratching my head trying to hook everything together lol).