VR Puppeteer to Puppet Input Transfer Problem in Unreal Engine
Overview
I’m a beginner in Unreal Engine working on a VR setup where a puppeteer’s inputs are transferred to a puppet in real-time. The goal is to have the puppet mirror the puppeteer’s actions, focusing on HMD and controller motions.
Setup Details
- Engine: Unreal Engine (version not specified)
- Plugin: Mimic Pro VR
- Characters:
- Puppeteer: Based on Unreal’s basic VR pawn
- Puppet: Based on the example character blueprint from Mimic Pro VR plugin, using the plugin’s full body IK Setup.
Current Approach
-
Blueprint Interface: Using a custom interface (Puppeteer_Puppet) to transfer motion controller references from Puppeteer to Puppet.
-
Reference Swapping: Attempting to ‘swap out’ the HMD and controllers in the Puppet with the Puppeteer’s.
-
Controller Setup:
- Puppeteer is possessed by the player controller
- Puppet is possessed by an AI controller (not currently utilized)
- Automatic Reference Setting: Im Mimic Pro’s default automated setting for assigning references to VR input- it worked before when i was possessing the puppet.
Puppeteer: The above image shows how I’ve set up the VR hardware inputs to the BP Interface in the Puppeteer’s Event Graph
Puppet: The above image shows how the interface was then used to create an event that outputs the VR hardware references, which are then ‘set’ as components
Current Behavior
- VR plugin correctly gets controller references for the Puppet (this worked even without Transferring references with BP interface)…
- HMD movement has no effect, despite the blueprint debugger showing that the Puppeteer’s HMD reference is being read.
Controllers are recognized, but not headset, causing upper body to collapse.
The Puppeteer’s HMD reference is showing as successfully set in the puppet BP during runtime debugging, but it still has no effect.
Desired Outcome
- Full transfer of HMD and controller movements from Puppeteer to Puppet.
- Eventually, implement full VR character functionality (pick up/drop, morph targets, physics modifications, control rig adjustments).
Challenges, Questions and Hypothesis
- Why isn’t the HMD movement transferring, even though the references seem to be correct?
- Is the current approach of reference swapping viable, or is there a better method?
- How can I ensure robust functionality beyond simple mirroring?
- Is it because the VR Plugin needs a player controller to work for some reason?
Additional Context
- The Puppet works perfectly when directly embodied (using default Mimic Pro plugin setup).
- Mimic Pro plugin has a mirror option, but I’m seeking a more robust solution for advanced use cases that can support things like grabbing and play well with the VR expansion plugin etc. I want the engine to treat it like a primary character. (But Maybe using this existing mirror solution is the best option??)
- I’ve been stuck on this issue for several days and would greatly appreciate any insights or alternative approaches.
- Here is the documentation for the Mimic Pro Plugin: Mimic Pro - Documentation
Finally, in case it helps, here is an image of the event graph from the VR body component from the Mimic Pro Plugin, that basically sets all the parts ofg the puppet up(VR hardware references, Skeletal mesh, animation blueprint, control rig etc). Note the delay at the start; I added it there so that im sure
THANK YOU FOR YOUR TIME AND HELP!!
-M