this is the discussion thread of my unofficial “Third Person C++ Perception Neuron” template and plugin which I have published for free under the MIT license on Github.
Perception Neuron ™ is a motion capture suit from Noitom financed by a Kickstarter campaign in 2014.
Further details about the suit here: https://neuronmocap.com
In the meantime also an official plugin version was released => Discussion thread.
Please use this thread to discuss the technical details about this unofficial template and plugin.
Live connection over TCP to Axis Neuon player or any other BVH server
Playback of locally stored BVH files with stop, pause and rewind
Support for multiple instances and different sources (can be shared)
Easy to use custom skeleton support with T-Pose, A-Pose or Pose with negative Y axis
(Just map bone names in event and anim graph. Since plugin v0.3.0 bone coordinate systems are mapped automatically)
No dependency to third party libraries and open source solution (MIT license)
Remove pelvis drift
In-Game and/or In-Editor animation recording (not part of free version, send me a PM if you are interested)
Changes Template:
v0.5.4:
Remove UE4.16 build warnings
v0.5.3:
Update to Plugin v0.3.3
v0.5.2:
Plugin enable Fix for UE4.14
v0.5.1:
Compile Fixes for UE4.13
v0.5.0:
Cleanup of all Animation BP and switch to new NeuronReadArray function which improves the support of custom skeletons.
Remove old custom bone mapping functions (are now deprecated)
Add function in Animation BP to remove pelvis drift on ground
Change project to UE 4.10
Update to Plugin v0.3.0
v0.4.0:
Network and player controller actors now support manual configuration of reference skeleton instead using external BVH reference file.
To have an example how to add custom skeletons, the Mannequin now uses the new bone map feature of the plugin.
For a “HowTo” please have a look into the Mannequin BP function “Init Bone Map”.
Update to Plugin v0.2.0
v0.3.0:
Source code restructure: C++ code is now placed in a plugin to be more project independend.
This plugin can be used in any of your projects and there is no need to create a C++ project anymore.
The binary plugin version 0.1.0 can be downloaded from here: https://github.com/Cyx69/TP_ThirdPer…ugin_0.1.0.zip
Further comments and minor improvements
v0.2.0:
Playback Controller added.
Now you can directly play BVH files placed ín your content folder instead connecting to a BVH server.
See new Blueprint nodes Play and Pause.
To have an example Neuron character is now connected to a player controller instead to a network controller.
Neuron and TPP_Hero are now part of a character blueprint (To have an example for that)
Hips bone of TPP_Hero mesh is used to translate complete actor (capsule). Actor rotation is still a problem and is not connected.
Add support for standard BVH format
Add support for disabled displacement and/or enabled reference bone in motion line
Add DNS name support
Add Blender source file for Neuron character
Further comments and minor improvements
v0.1.0: Initial version
Changes Plugin:
v0.3.4:
Remove UE4.16 build warnings
v0.3.3:
Do not resolve HostName if HostName is already a valid IPv4 address
v0.3.2:
Plugin enable Fix for UE4.14
v0.3.1:
Compile Fixes for UE4.13
v0.3.0:
Add better custom skeleton handling. No need to map bone coordinate systems by hand anymore, just map the bone names in anim and event graph.
Mark old neuron read functions as deprecated
Add BVH player rewind node
NeuronRead functions are now able to read the skeleton translation and rotation in component space
Compile pre build libraries with UE4.10
Further comments and minor improvements
v0.2.0:
It is now possible to initialize Reference skeleton manually and initialization with external BVH reference file is optional.
Support all rotation orders and add missing XZY, YZX, ZYX rotations
Better support for custom skeletons: Bone coordinate system can now be mapped in blueprint.
Fix a synchronization bug when using Neuron format
Further comments and minor improvements
v0.1.0: Initial version (based and included in template v0.3.0)
Unpack the ZIP archive to your UE4 template folder, for e.g. “C:\Program Files\Unreal Engine\4.9\Templates\TP_ThirdPersonNeuron”.
Make sure that the project files are placed and visible under “…\Templates\TP_ThirdPersonNeuron” after unzip.
Launch Unreal Editor and click on “New project”.
Under C++ you should now see a template called “TP Neuron”.
Create a new project with the above template selected.
Quick Installation Notes Plugin:
(Not needed if you use the template)
Configuration:
To connect from the template/plugin to the Axis Neuron SW following settings should be done in the Axis Neuron SW:
Open File|Settings dialog
Activate under Broadcasting BVH and choose port 7001
Configure under Output Format:
Frequency reducing: 1
Block type: String
Rotation: YXZ
Activate ‘Displacement’
Deactivate ‘Reference’
Per default the template/plugin connects to ‘localhost’ (127.0.0.1) and to port 7001.
You can change these settings in the Controller Blueprint if you want to use a separate PC or other ports.
And you can add further controllers if you want to retrieve motion data from additional suits or BVH server.
Donate:
If you find the template/plugin or parts of it useful, please support me and the further development and make a donation here: Donate
This is an amazing gift to the community and exactly what I was waiting for in order to preview a motion in the engine. Can’t wait to try it out. Thanks Cyx!
From a workflow perspective though how do you see this fitting into an animation pipeline? In its current form is its purpose simply to preview motion or do you see potential in it to grab a raw sequence directly in UE4 and start using it in Persona? (I would be blown away if we could already do this). As I understand it, and I could be wrong, you’d still need to save out the motion from Axis Neuron, do some clean up (which might include skeletal translation as I think all the bones have different names in Axis Neuron), and import the fbx into UE4. Does that sound right?
I’m still getting my head around the Axis Neuron -> UE4 workflow and would love to hear more views on it. Admittedly all I’m able to do at the moment is get some nice mocap in Axis Neuron and save it as fbx. I haven’t the foggiest how to apply it properly to the UE4 mannequin. When I tried to import the fbx onto the mannequin control rig it was a mess. I did manage to make it less messy by renaming the skeleton from the fbx to match the mannequin but figured there must be a better way/process.
@Naveed:
My focus so far was to get a live connection between UE4 and Axis Player and use it later also in a VR environment.
But I think it should be possible to store the values of the rotation and translation array for e.g. in a file and playback it later.
Some small enhancements in the Event Graph of the Animation BP should do it.
But may be there is a better way for your purpose if you have some Blender KnowHow:
Export your animation for instance in BVH format and import it in Blender. Apply a simple mesh to it, export it as FBX and import this to Unreal.
I got everything up and running in short order and was able to test some basic data that I captured earlier this week with one of the suites we have.
Other than some finger movement oddities it works very well. I’ll get around to doing some live capture tests later tomorrow but first thing tomorrow I’ll be sending you my donation. Great job ! You’ll certainly be hearing more from me shortly
Hi. can any of you guys show some videos of the perception neuron working? or give a short review? I am interested in this system but it is so new that there are not many reviews out there… how is the latency on the “live” set up… and do you have to do a lot of cleaning after you record something? Thanks!
catalejo, you can download the neuron software for free here: Downloads | Perception Neuron Motion Capture
it comes with several recordings that you can play back and test this template with.
well, it works like in the videos noitom posted
you just have to stay away from magnetic sources. e.g. I can’t use it at my desk, because there is too much interference from whatever (a lot of drift).
in the next room, the PN works flawlessly
I’m having a little difficulty getting my custom mesh to replicate the movement of the data stream. I’m using Blender btw
I followed the steps below , please do let me know if I’m on the right track and if I’m missing any steps that you may have followed yourself.
1- Export Neuron Mesh from Template as FBX.
2- Import into blender and replace Neuron Mesh with Custom Mesh, make sure that it’s rigged.
3- Export Custom mesh as FBX (I have a feeling that my settings may be off)
4- Import Custom Mesh FBX into UE4, Retarget Nueron Mesh’s AnimBP to Custom Mesh.
5- Go through New AnimBP resulting from Retarget, make sure that all the Transform(Modify)Bone nodes match my current skeleton’s bones.
6- Drop Skeletal Mesh into scene,assign the new AnimBP to it. Then open the Level Blueprint and set the reference in there.
Thus far it works with the exception that the movement is wildly off. The character is often upside down with arms and legs backwards etc, moving but not in the right way.
I would be most grateful if you could share the blender workflow you followed for preparing your custom mesh, what FBX settings you used for export and possibly what version of blender you used.
Thanks in advance !
you don’t need to export the Neuron Mesh, I have published the Blender source file for free here:
To add your mesh you can follow these steps in Blender:
Delete the old Neuron (Makehuman) mesh
Add your own mesh and tweak the mesh in that way that the armature fits into the mesh
Parent the mesh to the existing armature with empty or automated weighted groups (Whatever you like to work with)
Check all groups if they are present and correctly weighted
Use my Blender export settings that I described in der Blender.zip
Import the FBX to Unreal and choose the Neuron skeleton or create a new one if you prefer that
You could also manipulate the armature in Blender if you want and import it as a new skeleton in Unreal.
For instance the length of a bone, but then you have to switch the transform settings to ‘Ignore’ in the Animation graph for that bone.
But do not change the rotation of a bone and make sure that all Roll values for the bones are set to zero.
And if you use a new skeleton you may be have to check all ‘Transform (Modify) Bone’ nodes in the Anim graph if the ‘Bone to Modify’ setting is still correct.
Playback Controller added.
Now you can directly play BVH files placed ín your content folder instead connecting to a BVH server.
See new Blueprint nodes Play and Pause.
To have an example Neuron character is now connected to a player controller instead to a network controller.
**In other words: You can now play your recorded Neuron BVH animation files in Unreal. **
And you can also play animations created by other tools if the skeleton matches the reference skeleton (see axis.bvh file).
Neuron and TPP_Hero are now part of a character blueprint (To have an example for that)
Hips bone of TPP_Hero mesh is used to translate complete actor (capsule). Actor rotation is still a problem and is not connected.
Add support for standard BVH format
Add support for disabled displacement and/or enabled reference bone in motion line
Yea the news for BATCH E sucks but hey what can we do…
I’m fully supporting this project and will definitely contribute by donation.
Hopefully I’ll have my suit by mid Nov.
Man it’s crazy the time difference for delivering only a few hundred suits…
Almost an entire year to place about 1500 orders…
anyway cheers
and congrats
Outstanding work! Would I be correct in saying that with this update UE4 can handle BVH file types as well as it can FBX?
Cannot wait to try this when my neurons arrive.
@Naveed:
With this plugin it is now possible to play BVH motion capture files directly in UE4 that are stored somewhere in your content folder.
They are not directly imported like FBX files.
And you will need a skeleton that is supported by the plugin.
Currently supported are: Perception Neuron, TPP_Hero or Mannequin skeletons.
For other skeletons types the plugin must be patched manually at the moment.
If there is a need for other skeletons, please send me a PN or mail.
