[Plugin] Leap Motion - Event Driven

Development Programming & Scripting C++
, ,
366

mine failed, but i don’t known if its because the plugin or something wrong after upgrade to Visual 2015 since i update from 2014. Haven’t time for check.

edit:
compiled but i got this error on load

LogModuleManager:Warning: ModuleManager: Unable to load module ‘E:/Documents/Unreal Projects/xVR2/Plugins/LeapMotion/Binaries/Win64/UE4Editor-LeapMotion.dll’ because the file couldn’t be loaded by the OS.

edit2:
something wrong using duplicate project, i backup my project instead of duplicate for preserve 4.9.2 files, and work fine.

binary http://s000.tinyupload.com/index.php?file_id=08312723592589999963

367

is your plugin going to be built into one of the upcoming ue4 releases?

368

Update to 0.9.8
-Updated binaries to UE4.10

Image hands are still not ready atm.

Let me know if any bugs up!

Not that I know of :open_mouth:

My version compiled fine without any code changes, have a go with the new release (0.9.8) to confirm if it works.

369

There may be a subjective difference betweeen the yellow character and the echo hands since the echo hands are closer to 1:1, but collision itself should not make a difference, it is literally a single toggle.

BTW Echo hands are Leap assets that I’ve gotten permission to release with my plugin, so praise leap for the geometry :slight_smile:

Update to 0.9.9
-Updated Mac binaries to 4.10

Note that mac binaries will be very rarely synced due to lack of time, so I highly urge a Mac user of the plugin to take the helm for keeping mac binaries in sync.

Edit: updated this response with 0.9.9 change

Mac is now synced to 4.10, but you have full source so you can compile the mac binary yourself at any time and make a pull request to include the binaries in the master branch for others.

370

Hi,

I just tried 0.9.7 on MAC 10.10.4 + Leapmotion 2.3.1 + Unreal 4.9.2

But always report “Plugin ‘LeapMotion’ failed to load because module ‘LeapMotion’ does not appear to be compatible with the current version of the engine. The plugin may need to be recompiled.” after I copy binaries and plugin to the project folder.

What is wrong?

BR

371

0.9.7 has mac binaries synced to 4.7. Use UE4.7 or use plugin v 0.9.9 with UE4.10. Optionally add code to your project and recompile as you have source included with the plugin.

372

Hi !

First let me thank you for all the plugins you’ve been sharing on .

I was wondering if you had any idea on how I should approach translating your bp using only c++.
I mean reproducing the logic of your blueprints such as RiggedCharacter or VRController in code.
Been playing around with the plugin, but thought I would ask you directly if you’d thought about it already.

Edit:
It seems that using directly the plugin in c++ was not possible based on one of your earlier answer (#45]([Plugin] Leap Motion - Event Driven - C++ - Epic Developer Community Forums)).
Is it still the case? :slight_smile:

Edit2:
Based on the workaround from your previous answer, I duplicated and then modified the parent of LeapBasicRiggedCharacter for a c++ character.
It does work and allow to have the c++ code of the character and the leap plugin character working in concert.
But in such an implementation, how could we access the leap info such as the hand position from the code?
Does it forces us to do everything from blueprints with the plugin?

373

Glad my plugins can be of use!

To access leap motion from C++, do the same thing you would do in blueprint but using the C++ methods. If you’re confident you should probably use the custom approach as it will give you more direct control and flexibility which is what I’m assuming your C++ approach is about.

  1. Attach a component to your pawn/actor.

  2. Use the LeapEventInterface via multiple-inheritance

Now you will receive the same events you would get in blueprints, all defined in the LeapEventInterface class. Then you can follow the same instructions found at the repository readme, pay attention for example to the Example ‘Debug Hand’ graph on how to get basic hand information.

You have nearly complete access to the Leap API through the UE components which handle all orientation and unit conversion, but since you’re in C++ you can also query the leap directly after including the necessary headers, e.g. using the pure Leap API Leap Controller


Leap::Controller leap;

See my unreal ULeapController component class for use examples.

On the topic of recreating the VRController, all it does is ensure that your camera is separated from your view controller rotation. You can follow this wiki for an example of how to do that in C++.

All the convenience characters such as RiggedCharacter are a bit more complex. What they do is contain an actor called LeapAnimBodyConnector or its subclass, which handles all the leap-specific interaction and stores it in an AnimBody data structure class. Then in the Animation Blueprint of the skeletal mesh, it looks for the AnimBody and copies its current state to the skeletal mesh, blending in animation when information is not being sent. This means the animation blueprints and characters are agnostic to the input, and the only leap specific part is the LeapAnimBodyConnector.

I haven’t had the opportunity to explore how animation blueprints can be set in C++, if you find out how to do that, some guidance on it would be awesome and might help pave the way for easier skeletal mesh swapping in the future.

374

Thank you for the detailed answer! :slight_smile:
I will give it a try tonight and let you know how it goes.

375

I tried to add the LeapController component to a default character, but so far I can’t managed to link the code.

In the Char.h I include “LeapController.h” and define a pointer to the component.



class ULeapController* LeapController;

In the Char.cpp, I instantiate the LeapController in the class constructor



ALeapCharacter::ALeapCharacter()
{
	PrimaryActorTick.bCanEverTick = true;

	LeapController = CreateDefaultSubobject<ULeapController>(TEXT("LeapControllerComponent"));
}

But end up getting the following link error:


LeapCharacter.cpp.obj : error LNK2019: unresolved external symbol "private: static class UClass * __cdecl ULeapController::GetPrivateStaticClass(wchar_t const *)" (?GetPrivateStaticClass@ULeapController@@CAPEAVUClass@@PEB_W@Z) referenced in function "public: class ULeapController * __cdecl UObject::CreateDefaultSubobject<class ULeapController>(class FName,bool)" (??$CreateDefaultSubobject@VULeapController@@@UObject@@QEAAPEAVULeapController@@VFName@@_N@Z)

Is there another step I’m forgetting about?

Edit:
It seems that adding the MinimalAPI tag for the ULeapController class fixes the compilation error to instantiate the component.



UCLASS(MinimalAPI, ClassGroup=Input, meta=(BlueprintSpawnableComponent))
class ULeapController : public UActorComponent

But the same kind of linking error happen when using a method of ULeapController.
Example when using the EnableBackgroundTracking:


LeapCharacter.cpp.obj : error LNK2019: unresolved external symbol "public: void __cdecl ULeapController::EnableBackgroundTracking(bool)" (?EnableBackgroundTracking@ULeapController@@QEAAX_N@Z) referenced in function "public: __cdecl ALeapCharacter::ALeapCharacter(void)" (??0ALeapCharacter@@QEAA@XZ)

Using the LEAPMOTION_API tag for the class definition of ULeapController does seem to export all methods.


class LEAPMOTION_API ULeapController : public UActorComponent

Is it intended?

376

I haven’t touched C++ project binding to plugins in a while, and I’ve just recently hit the same problem actually with my zip plugin!

I’m going to try your fixes and get back to you, but thanks a lot for the suggestions! If this compiles for you without problems, consider making a pull request to have the changes added to the master branch.

Edit: Back in 4.0 I would copy the class implementation files for editor (where plugins are isolated in their own dlls) and remove them in packaged solutions (your dlls are collapsed into a monolithic build so it would link without issue). So this is much cleaner than that hack.

According to plugin documentation what we’re doing is essentially creating a hard bind, but I believe it falls under

If you are creating a game plugin (not engine plugin), and you want to allow the game to statically link against one of your plugin modules. This breaks the concept of a plugin a little bit, but is often useful for plugins that want to declare new UObject types that game classes can inherit from or use directly. The engine itself has no dependencies on this plugin, but the game project code and content may very well have direct dependencies.

I’m not 100% sure on best practices, but it does seem to be the most convenient and when you want to use leap motion specific methods in C++ there really isn’t a practical way to not use this approach.

377

Finally had a bit of time between the exams to work on this.

It seems that the only problem there for the leap motion plugin was the lack of the export macro on the classes or specific methods.
Might be the same for the Zip plugin.

It seems that best practices are fuzzy for many people on many topics… :smiley:
Hopefully, more will pop up in the documentation.

In the meantime, I decided to keep going into the conversion to C++.
In short, it works but needs to be finished.
Most of the blueprints functionalities are now also available from C++ but can also be manipulated via blueprints. This means that you can create a game mode using directly the c++ LeapCharacter class in the settings. The VRController will come later.
The only blueprints really necessary to conform with the “best practices” is the animation blueprint.
And since I’m lazy, I preferred adding custom animation nodes than connecting each bones of each fingers to all the nodes… I have to say, I’m impressed by your patience to make it even all look nice in there. :stuck_out_tongue:

More information in the commit message: Commit details

There’s still some work to do, but it should allow both the blueprint and c++ ways to coexist at each level.

378

Just had a look at the commit, that’s a beefy commit, great work!

I should have a chance to play with it this weekend, always wanted a better understanding of setting up IK/animation in c++ and I think this is a great resource to brush up on that department.

Long term thoughts:
In the end I think convenience setups such as rigged characters should be abstracted away from the hardware plugins themselves. A full ‘abstract skeleton’ target could be used that can take input from multiple devices and merge the information by the best data available. From that skeleton you would then extract the information and forward it to any output node e.g. a convenience rigged character. Then whether you use just floating rigged hands, poly models or a full body character you would have the same backend and you wouldn’t need to change your rigging for new hardware inputs and you would now support mixed inputs. In my mind this is something akin to a Body State Plugin. You could further expand this by adding some smart class to handle skeleton re-targeting, which could automatically match various skeletal meshes to the abstract one and perhaps allow for almost a one click change to swap any rigged character to support a new skeletal mesh you want to replicate your movement. That would be the ideal setup in my mind.

The hard part of this is figuring out a way in which plugins such as this one can forward input automatically to a target that may not be there (unless something like this idea makes it into the engine and we can depend on it existing).

379

We might see such an interface happen in the near future.
There’s quite a lot going on around VR, and I understand that teams like Epic have other priorities than body rigging for now.
But it could make quite an interesting project during my thesis if I still have time after the original topic. :wink:

Have you had the time to take a look at how I took care of the fingers bones?
Code is still a bit messy since it was my first Unreal Engine coding session, but I could clean it up and comment more seriously if you have any use for it. Same for the general c++ implementation, if it could somehow be useful, we can discuss the architecture to integrate it to the master.

By the way, have you played with the “image-hand” that Leap integrated to their unity3D asset?
I didn’t see it in their UE4 plugin.

380

I like the way you handle as much as possible in C++ and your overall structure is good. I think C++ anim blueprint nodes are the way to go allowing for cleaner setups downstream if we can handle input diversity well (orientation only FK (e.g. multiple IMUs) vs positional tracking IK (e.g. motion controllers)). The plugin as it is, is due for a refactoring and I would love to incorporate these changes as well start the Body State separation so all input plugins could target the same animation chain. I’ll pm you if you’re interested in helping/carrying this further.

The end goal in my mind would be to allow a developer to select a skeletal mesh with humanoid like rig and just select e.g. map Body State to this skeleton and have the plugin do all the hard work of re-targeting if necessary and switch animation chain to handle the type data available (do we have orientation only? Hands only or full body? Mixed inputs? which one is more accurate for this frame?).

I have and this plugin’s UE version is in the works as we speak. It’s fairly close to being ready, but there are quite a few changes to get right and given the fully deferred renderer of UE4, it may not look exactly the same. One of the non obvious aspects of image hands is that you have to use the leap IPD (40mm) for the hand look and interact with the world as you would expect which makes everything feel a bit bigger.

381

Does this work with the Orion SDK?

382

Yep just update your runtime. The only thing the plugin won’t have is the low latency update from closer to metal api, will need to wait for an api update before I can have a look at it.

383

Hi, I’m the guy with the axis problem. I’m new with leap and with Ue4 so im very lost. I want to do something like this v=am6Rt2toZZc
but it looks like that: Falla - YouTube [I have not much idea of what i’m doing] so i want to ask you if you can please, please show me how?
Thanks for all.

384

I think the low latency feeling only comes from the “temporal warping bias” that is presented as an option. If I put it to 0 I see the same “lag” as in UE4. I thought about doing something like this for some time and now I have the motivation as it makes quite a difference.
I plan to get the relative position from hands to headset and move them to where they were relative to the HMD x ms before.
How can I get to do that at each update hands event so that it doesn’t add-up.

I know it’s not very inteligible, I’ll try to add a blueprint or some scanned sketches.

386

Hello!

I’m currently using the plugin for leap motion in unreal engine 4 in combination with occulus rift dk2 to make a game. The game will be played on a virtual table so the players should be able to reach everywhere on the map (it should feel like you are sitting around a real table). In order to achieve this I want to change the “world to meters” parameter in world settings under VR, but when i increase it to for example 200 (default is 100) the leap motion hands start behaving weird. A cross eye effect occurs and if I increase it even more the leap hands eventually disappears completely. One way to solve it is to keep the world to meters parameter as it is and just scale the game world down, but this resulted in issues with the navigation mesh as I had to scale everything down a lot.

So my question is if this problem when increasing the “world to meters” parameter is at all connected to the leap motion plugin or is it some occulus/ue4 specific problem? Or am I missing something obvious here? Haven’t found anyone with the same problem so im starting to suspect that it might be something fundamental im missing. It’s my first time working with ue4, occulus and leap motion so everything is kind of new.