[Plugin] Leap Motion - Event Driven

Hi,
I’ve created a new project to test the 4.9 version of the pluging but when I select LeapBasicRiggedCharacter for Default Pawn Class in World settings my project crash, same thing for the other options : LeapBasic…LeapCollision…
I’ve copied Binaries and Plugins folders in the root as mentioned in the documentation

any help please ?

Thank you

Is this from a fresh project? what are the steps to reproduce this issue?

Did you add a component and interface as outlined in the readme? If you did then you should be able to just hit compile on the blueprint and right click to see all the leap specific events. If you have issues let me know and I’ll look into it.

I was just going to edit my post to add exactly that :wink: . My bad. I the whole page again today (sloooowly) and noticed the interface instruction.it’s all working now :slight_smile: .
Can I suggest you BOLD or Underline the “add Interface” part. It’s easy to miss.

Thanks so much.

btw LM & Myo owe you big time ! I would have “never” known about Myo had I not stumbled upon your Myo thread (ironically I noticed the “addInterface” part there and realised i may have made the same mistake with setting up LM).

@EPIC hire this man already !

Yes a fresh project. I created a new one then placed the Binaries and Plugins folders ( from the 's zip, verion 4.9 ) in the root
I created a game mode blueprint then tried to assigne LeapRiggedCharacter to the Default Pawn Class but everytime the project crashs. All the other options Leap*** produce a crash except LeapBasicRiggedCharacter which generate this error :

testing the 4.9 update and it runs much smoother than the previous versions. :slight_smile:

But here is an issue i ran into, and I cannot figure out how to solve this properly:

I want to have the camera follow the animation. So when a different animation is played e.g. “sit”, the camera stays at the same position relative to root.
I figured attaching the camera to the head socket would solve this, which it does.
However the LeapConnector must also placed relativ to camera/head to work correctly.
So this is my setup:
736f4037293e48649b96c7097fe04803ea83093b.jpeg

Now when running VR mode, at the moment hands are tracked, this happens:
9c4bcd98a10fc9d6ae53373dcd05caa37ee25765.jpeg

Is my approach wrong or must I change something in the connector/anim?

Hmm weird, just a sanity check you’re using Windows right? Only windows platform has 4.9 support, Mac is currently only compiled to 4.7, and Linux is unsupported.

This is an interesting problem and it has everything to do with how rigging is currently set up. The character is rigged such that you do not get micro motions from animation while still maintaining relative head offset and causing the body to turn and twist with your body leaning. It is a set of compromises to fit all the tracking criteria and may not be suitable for all applications.

In VR, it is generally a bad idea to add animation motion to your hmd as it will encourage motion sickness, so if you wanted someone to sit you would play the sitting animation and keep the head IK(from camera/animbody) in the rough area where your head is while sitting. Your head mesh is automatically hidden so small mismatches won’t be visible, large mismatches will however have the character bend in unrealistic ways like the one you have where the head height is much lower than normal.

All animbody positions are given relative to the character root with Y-forward orientation, you may need to adjust this if you want a different kind of functionality. You could have the animation centered around the head not moving, or you could change the rigging to accommodate what you want to achieve. I haven’t personally done a sitting character so I don’t have a ready to go solution for this…

Full character rigging in VR is a complex issue with compromises abound; a standard approach is to not have a body for this reason e.g. you can use the Echo hands character and not worry about fully body rigging at all, but I encourage you to explore and share your solutions so we can have one more thing solved in VR :slight_smile:

yes windows 8.1 and UE4.9.0
I just tried a new blank project, no starter content, and the problem persists :frowning:

This shouldn’t fix it, but just in case update to 4.9.1 and try again.

Then pm me your logs from the crashes, found under <Project Root>/Saved/Logs

I can’t find an option to attach the log file in a PM :confused:

I checked the log then found this :

[2015.10.02-16.28.14:539] 20]LogLinker:Warning: Asset ‘…/…/…/…/…/…/Users/Admin/Documents/Unreal Projects/TestGetnamo2/Plugins/LeapMotion/Content/Character/Basic/TPPModular_Skeleton.uasset’ has been saved with engine version newer than current and therefore can’t be loaded. CurrEngineVersion: 4.9.0-2672455+++depot+UE4-Releases+4.9 AssetEngineVersion: 4.9.1-2672455+++depot+UE4-Releases+4.9
[2015.10.02-16.28.14:539] 20]LogLinker:Warning: Asset ‘…/…/…/…/…/…/Users/Admin/Documents/Unreal Projects/TestGetnamo2/Plugins/LeapMotion/Content/Character/Basic/TPPModular_Skeleton.uasset’ has been saved with engine version newer than current and therefore can’t be loaded. CurrEngineVersion: 4.9.0-2672455+++depot+UE4-Releases+4.9 AssetEngineVersion: 4.9.1-2672455+++depot+UE4-Releases+4.9
[2015.10.02-16.28.14:540] 20]LogLinker:Warning: Asset ‘…/…/…/…/…/…/Users/Admin/Documents/Unreal Projects/TestGetnamo2/Plugins/LeapMotion/Content/Character/Basic/TPPModular_Skeleton.uasset’ has been saved with engine version newer than current and therefore can’t be loaded. CurrEngineVersion: 4.9.0-2672455+++depot+UE4-Releases+4.9 AssetEngineVersion: 4.9.1-2672455+++depot+UE4-Releases+4.9
[2015.10.02-16.28.14:540] 20]LogLinker:Warning: Asset ‘…/…/…/…/…/…/Users/Admin/Documents/Unreal Projects/TestGetnamo2/Plugins/LeapMotion/Content/Character/Basic/TPPModular_Skeleton.uasset’ has been saved with engine version newer than current and therefore can’t be loaded. CurrEngineVersion: 4.9.0-2672455+++depot+UE4-Releases+4.9 AssetEngineVersion: 4.9.1-2672455+++depot+UE4-Releases+4.9
[2015.10.02-16.28.14:542] 20]LogAnimation:Warning: Invalid animation root node index 54 on ‘/LeapMotion/Character/Basic/BasicCharacter_AnimBlueprint.BasicCharacter_AnimBlueprint_C’ (only 52 nodes)
[2015.10.02-16.28.14:543] 20]LogLinker:Warning: Asset ‘…/…/…/…/…/…/Users/Admin/Documents/Unreal Projects/TestGetnamo2/Plugins/LeapMotion/Content/Character/Basic/TPPModular_Skeleton.uasset’ has been saved with engine version newer than current and therefore can’t be loaded. CurrEngineVersion: 4.9.0-2672455+++depot+UE4-Releases+4.9 AssetEngineVersion: 4.9.1-2672455+++depot+UE4-Releases+4.9
[2015.10.02-16.28.14:543] 20]LogLinker:Warning: Asset ‘…/…/…/…/…/…/Users/Admin/Documents/Unreal Projects/TestGetnamo2/Plugins/LeapMotion/Content/Character/Basic/TPPModular_Skeleton.uasset’ has been saved with engine version newer than current and therefore can’t be loaded. CurrEngineVersion: 4.9.0-2672455+++depot+UE4-Releases+4.9 AssetEngineVersion: 4.9.1-2672455+++depot+UE4-Releases+4.9
[2015.10.02-16.28.15:050] 20]LogCrashTracker:

I’ll upgrade then let you now

thank you

So that could be it, it looks like blueprints are sensitive to minor build versions. Good to know for the future!

Hi,
not able to get the UE 4.9.1 from the launcher I installed the 4.9.2 and get this issues in the log file :

[2015.10.05-09.37.54:176][233]LogAnimation:Warning: Invalid animation root node index 54 on ‘/LeapMotion/Character/Basic/BasicCharacter_AnimBlueprint.BasicCharacter_AnimBlueprint_C’ (only 52 nodes)
[2015.10.05-09.37.54:459][233]LogAnimation:Warning: Invalid animation root node index 54 on ‘/Engine/Transient.REINST_BasicCharacter_AnimBlueprint_C_0’ (only 52 nodes)
[2015.10.05-09.37.54:577][233]LogClass:Warning: Property AnimGraphNode_Fabrik_8C904C7A415FB588652E8A91FDC811AE of BasicCharacter_AnimBlueprint_C has a struct type mismatch (tag FallbackStruct != prop AnimNode_Fabrik) in package: FObjectReader. If that struct got renamed, add an entry to ActiveStructRedirects.
[2015.10.05-09.37.54:577][233]LogClass:Warning: Property AnimGraphNode_Fabrik_416EC86F435836563C4E49AD6D016C3C of BasicCharacter_AnimBlueprint_C has a struct type mismatch (tag FallbackStruct != prop AnimNode_Fabrik) in package: FObjectReader. If that struct got renamed, add an entry to ActiveStructRedirects.
[2015.10.05-09.37.54:577][233]LogClass:Warning: Property AnimGraphNode_Fabrik_9FF4326C4CAD2373B92D6F9E3AC3F57A of BasicCharacter_AnimBlueprint_C has a struct type mismatch (tag FallbackStruct != prop AnimNode_Fabrik) in package: FObjectReader. If that struct got renamed, add an entry to ActiveStructRedirects.
[2015.10.05-09.37.55:001][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning InputAction Event references unknown Action ‘Jump’ for InputAction Jump
[2015.10.05-09.37.55:002][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘LookUpRate’ for InputAxis LookUpRate
[2015.10.05-09.37.55:002][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘TurnRate’ for InputAxis TurnRate
[2015.10.05-09.37.55:002][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘Turn’ for InputAxis Turn
[2015.10.05-09.37.55:002][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘LookUp’ for InputAxis LookUp
[2015.10.05-09.37.55:002][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘MoveRight’ for InputAxis MoveRight
[2015.10.05-09.37.55:002][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘MoveForward’ for InputAxis MoveForward
[2015.10.05-09.37.55:006][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning [0057.68] Compile of LeapBasicRiggedCharacter successful, but with 7 Warning(s) [in 13 ms]
[2015.10.05-09.37.55:007][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning [0057.68] Compile of LeapBasicRiggedCharacter successful, but with 8 Warning(s) [in 0 ms]
[2015.10.05-09.37.55:007][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning [0057.68] Compile of LeapBasicRiggedCharacter successful, but with 9 Warning(s) [in 0 ms]
[2015.10.05-09.37.55:020][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning InputAction Event references unknown Action ‘Jump’ for InputAction Jump
[2015.10.05-09.37.55:020][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘LookUpRate’ for InputAxis LookUpRate
[2015.10.05-09.37.55:020][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘TurnRate’ for InputAxis TurnRate
[2015.10.05-09.37.55:021][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘Turn’ for InputAxis Turn
[2015.10.05-09.37.55:021][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘LookUp’ for InputAxis LookUp
[2015.10.05-09.37.55:021][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning Input Axis Event references unknown Axis ‘MoveRight’ for

Not sure what the first warnings are and I will have a look at the invalid animation node one.

You can safely ignore these type of warnings:
*[2015.10.05-09.37.55:001][233]LogBlueprint:Warning: [compiler LeapBasicRiggedCharacter] Warning InputAction Event references unknown Action ‘Jump’ for InputAction Jump *

this is emitted simply because the character is bound to the standard move events defined in third person templates. If they’re not present, WASD and other movement inputs will simply not do anything unless you link them up in a way you want the external IM to work, see third person template for example of how.

I decided to stick to the original setup.

Now I’ve got the next issue.
The hands apparently are offset to the debug spheres.
Why is that and how can I fix this?
If the bones matched the spheres perfectly or at least more closely, very accurate tracking would be possible.
The way it is now, overlaps may happen breaking some immersion.

74bf4ad9a0561f8e4f797c746de2c470a9ac72ae.jpeg

Can you supply 4.9 Binary file for mac?

Is it possible to use this plugin in hmd mode in a ue 4.9 project where I already have a c++ character and playercontroller?

There are two possible reasons for this. One, if you don’t have a VRPlayerController any hmd rotation may desync your rigged hands since your controller orientation is not decoupled from your hmd. The second is that the rigging uses compromises to match a mesh to your physical dimensions. It resizes the mesh arm length to match yours and this causes the hand to shrink, if this wasn’t the case the hands would deviate more. If you want a more 1:1 rigged experience try the floating LeapEchoRiggedCharacter, which without needing to adjust elbows and such the rigging is able to match closer to the debug hands.

If you can personally modify the rigging to have a better compromise, by all means make a pull request :slight_smile:

If you have a Mac, you should be able to just compile the plugin. You have full source and should be able to recompile it for 4.9 without issues. That said I’ll include a final 4.9 mac binary when image hands make it in to the plugin (Mac isn’t my main platform and I have an ancient Mac so unless someone takes up the Mac compile mantle it will always lag significantly behind the windows releases).

Yes, all you do is follow the instructions in the repo (and earlier in this thread) found under the How to use it - Blueprint without Convenience Content, Quick Setup heading. There you can forward/store all the latest leap motion tracking points to where you want to use them. You can also look into the LeapBasicRiggedCharacter blueprint to see how the more complicated rigging is handled using the intermediate AnimBody. Note that if you do not use the convenience characters, you will have to set your hmd mode from the component manually to be triggered when the hmd gets enabled (Optimize for HMD function).

once I add the plugin to 4.9.2
I can’t package the project

It’s said that
MainFrameActions: Packaging (Windows (64-bit)): BuildCommand.Execute: ERROR: BUILD FAILED
MainFrameActions: Packaging (Windows (64-bit)): Program.Main: ERROR: AutomationTool terminated with exception:
MainFrameActions: Packaging (Windows (64-bit)): Program.Main: ERROR: Exception in AutomationScripts.Automation: Cook failed.
MainFrameActions: Packaging (Windows (64-bit)): Stacktrace: 在 Project.Cook(ProjectParams Params)
MainFrameActions: Packaging (Windows (64-bit)): 在 BuildCookRun.DoBuildCookRun(ProjectParams Params)
MainFrameActions: Packaging (Windows (64-bit)): 在 BuildCommand.Execute()
MainFrameActions: Packaging (Windows (64-bit)): 在 AutomationTool.Automation.Execute(List1 CommandsToExecute, CaselessDictionary1 Commands)
MainFrameActions: Packaging (Windows (64-bit)): 在 AutomationTool.Automation.Process(String] CommandLine)
MainFrameActions: Packaging (Windows (64-bit)): 在 AutomationTool.Program.MainProc(Object Param)
MainFrameActions: Packaging (Windows (64-bit)): 在 AutomationTool.InternalUtils.RunSingleInstance(Action`1 Main, Object Param)
MainFrameActions: Packaging (Windows (64-bit)): 在 AutomationTool.Program.Main()InnerException in AutomationUtils.Automation: BUILD FAILED: Failed while running Cook for E:\Unreal Engine Projects\MyProject_9\MyProject_9.uproject; see log C:\Users\Administrator\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+4.9\Cook-2015.10.20-15.30.52.txt
MainFrameActions: Packaging (Windows (64-bit)): Stacktrace: 在 AutomationTool.CommandUtils.RunCommandlet(String ProjectName, String UE4Exe, String Commandlet, String Parameters)
MainFrameActions: Packaging (Windows (64-bit)): 在 Project.Cook(ProjectParams Params)
MainFrameActions: Packaging (Windows (64-bit)): ProcessManager.KillAll: Trying to kill 0 spawned processes.
MainFrameActions: Packaging (Windows (64-bit)): Program.Main: AutomationTool exiting with ExitCode=Error_UnknownCookFailure
MainFrameActions: Packaging (Windows (64-bit)): Domain_ProcessExit
MainFrameActions: Packaging (Windows (64-bit)): copying UAT log files…
MainFrameActions: Packaging (Windows (64-bit)): RunUAT.bat ERROR: AutomationTool was unable to run successfully.
MainFrameActions: Packaging (Windows (64-bit)): BUILD FAILED
PackagingResults:Error: Error Unknown Cook Failure

Hi, would the latest UE4.9 version of the plugin work with Oculus runtime 0.7.0 ?

Cheers

Hi
I surely don’t want to rush you and I’m still impressed with your plugin.
Do you think you’ll have time to update the plugin to 4.9.2 cause I must have a build ready by the 28th to meet some clients…
If you’re too busy that’s ok I just have to figure how to rebuild version UE4.9.1 from (I can only build version 4.10.0…)

Thanks your amazing!

I’ll fix the packaging this weekend.

Yep, UE4.9 uses the 0.7 runtime.