Just wrote an article on using UE4 with the new SteamVR Input System, plenty of tech info there for those interested:
Will also be doing a Livestream on this in my YT channel tomorrow at 11:15 AEST:
Wow, great work runeberg. Thanks so much for sharing all this!
@runeberg Thanks for this! Do you know of a way (or a simple node) to know if the user is using the Wands or the Knuckles?
not sure if there’s one in BP, but I detect Knuckles from C++ using this (admittedly rather clunky) way:
...
if (SteamVRSystem && trackedDeviceClass == ETrackedDeviceClass::TrackedDeviceClass_Controller)
{
uint32 StringBytes = SteamVRSystem->GetStringTrackedDeviceProperty(id, ETrackedDeviceProperty::Prop_ModelNumber_String, buf, sizeof(buf));
FString stringCache = *FString(UTF8_TO_TCHAR(buf));
//UE_LOG(LogKnucklesLivelinkSource, Display, TEXT("[KNUCKLES LIVELINK] Found the following device: %i] %s"), id, *stringCache);
// Check if Knuckles is present and active
if (stringCache.Contains(FString(TEXT("Knuckles"))) &&
SteamVRSystem->GetControllerRoleForTrackedDeviceIndex(id) == ETrackedControllerRole::TrackedControllerRole_LeftHand
)
{
KnucklesControllerIdLeft = id;
if (SteamVRSystem->IsTrackedDeviceConnected(KnucklesControllerIdLeft))
{
bLeftKnucklesPresent = true;
//UE_LOG(LogKnucklesLivelinkSource, Warning, TEXT("[KNUCKLES LIVELINK] Knuckles Left found and is ACTIVE"));
}
}
...
Hope that helps!
Just released a new version of SunRock which includes support for latest OpenVR (1.2.10) and is available for users in the project level (no engine recompile necessary). Also a utility script there to help update stock UE4 engine’s OpenVR SDK.
Exposed Splays & Curls in the Controller and created a couple of demoes for it (Gesture Recognition, UE4 Hand Mesh streaming anim).
Hope its helpful!
Changes vid:
Repo here:
https://github.com/1runeberg/SunRock
Thanks @runeberg I’ll see if I can use this. If I have any success in Blueprints I’ll share. Also thanks for the update to SunRock. Just getting into it.
A far cleaner method here, it’s Unity C# but the base api call is from the C++ api… Will update the plugin with this:
https://twitter.com/TheShadowBrain/status/1092433693174575104?s=19
That is a much better approach than prior, grats! It was very bloated with all of the extra controller setup before.
You don’t need to directly load the .dlls though anymore (in your modules primary source file), that is legacy behavior that they never switched away from back in an earlier version of steam that conflicted with the OpenVR module, you can directly reference it now just by calling into.
vr::IVRInput * VRInput = (vr::IVRInput*)vr::VR_GetGenericInterface(vr::IVRInput_Version, &Initerror);
As long as your include pathway for your module points to the new OpenVR.h source and not the engines than it should all work fine without conflicts (as of three or four engine versions ago I think, they just never updated the in engine module to change that).
I would also suggest that you stop naming everything knuckles too, for instance the finger splay values aren’t used for knuckles controllers and are there to future proof for controllers later on, also there may be multiple controllers that return the finger curl values in the near future. Its a generic input system from steam, not a knuckles specific one.
Throwing the curl / splay events as seperate input events also may be a bit overkill, considering that they generally are used in some combination or another and there is some overhead pushing all of those each tick to the graph. It also makes it more difficult to gather them together, maybe add an accessor to the function library that just returns the arrays? Then they also won’t be polled unless needed.
I was holding off on doing something like this as really all that the engine needs is the static skeletal actions to be added to the manifest…but that doesn’t appear to be happening. I’ll point people to your input device module to work with my skeletal controllers, there really shouldn’t be multiple versions of the same thing floating around if possible and I think that this newest version is good enough to not require any other methods.
Good work, this will save me a ton of time doing the same thing, I have a ton on my plate and appreciate not having to duplicate effort.
Hi Mordentral!
Thanks for the feedback and input, will take them on board.
This is the first iteration to “professionalise” the plugin for general public consumption as it’s now apparent that in-engine support won’t be coming in anytime soon, so am really glad to hear that you are on-board with the archi, I’ve mulled over a couple of designs taking into account possible future moves by both Epic and Valve along with expediency requirements, this I felt is a good “middle ground” and hopefully future proofs it a bit.
Need to profile the curls and splays as it is one of my concerns as well on the current implementation, I think I have a todo tacked on that bit of code actually. Agree on making the naming more generic as well, it’s a bit of a balancing act at the moment to ensure any naming won’t clash with whatever Epic is planning - I tried to stay as close as possible to traces of support in 4.21, even calling them “Grips”, but may need to diverge at some point based on direction of SteamVR.
Glad to have more devs playing with it and providing feedback/bugs/etc and certainly PRs directly to the code would be awesome.
Cheers from Down Under,
Rune.
Well, really just having the skeletal actions in the base beta input manifest would be enough. But I have several pull requests for VR already sitting around for a decently long period now without any attention so I kind of gave up on that one. It would be nice to not have the default implementation sitting around ticking needlessly though…
I just am loath to make yet another manifest generator between yours, the engines, and valves, this new setup is generalized enough (aside from the curl/spaly values) that people can use the manifest generation part fairly cleanly alongside different skeletal systems.
Hey runeberg, thanks a lot for the great plugin and project.
One question. How can I change the animation stream from VRSkeletalMotionRange_WithoutController to VRSkeletalMotionRange_WithController?
Hi Alphatierchen!
In the BP node “Connect With Knuckles Source”, there are two tick boxes:
“LeftWithController”
“RightWithController”
If you enable either one or both of them, you’ll get the WithMotionController SkeletalRange data streaming in.
In C++, it is controlled by the following public vars in KnucklesLiveLinkSource
bRangeWithControllerL
bRangeWithControllerR
Hope that helps!
Perfect! Thank you!
Hi Runeberg,
Congrats for this amazing work, It’s really a big add to steamvr plugin.
I’m concern about the use of vive trackers (used with pogo pins) with your plugin, as I’m dealing with losing action mapping with steamvr input binding. Can your plugin be used with vive trackers and will it generate input profiles in order to avoid using the steamvr input binding ?
Thanks
Maxime
Hi Maxime,
Thanks for bringing this up and apologies for the late reply. At the moment, there’s no support for the pins, but please watch this thread in the next days to get some progress updates 
Any other suggestions and/or feedback, please just let me know.
Cheers,
Rune.
HI Everyone!
Just wanting to announce that from today, I’m discontinuing support for this plugin as I’ll be working with Valve on the official Unreal Engine plugin.
Hope to have some more announcements on it soon. Any questions and/or feedback that you’d liek to see on the official plugin, please jsut let me know!
Cheers,
Rune.
Hey Rune,
this is good news! An official plugin is very much needed. We have been toying around with our Knuckles and your plugin, however, knowing we’ll have to do all again when a official plugin comes out, kept us back.
This news is much appreciated.
Looking forward to hear more, especially timeline, do you need betatesters, and more. EV3 and DV1 available.
Cheers
Just tried your SteamVRInput plugin. Great work! I need it to use Knuckles in my game along with other controllers, so I modified the plugin a bit to add support of legacy UE4 VR inputs. As you now work at official plugin, here is the list of improvements I’d like to see in it (ha-ha):
- Support for multiple actions from the same input path (I now save them in FInputMapping array and handle manually in FSteamVRInputDevice::SendActionInputEvents, but probably Valve could add it to their binding json format?)
- It wouldn’t be a problem if UE4 has new input system enabled by default, but for now it isn’t so. Input settings of one project are shared among all projects in development, because action manifest of all projects are associated with UE4Editor.exe. As a result, VR input doesn’t work when I switch to a project without SteamVRInput plugin.
- Input events from Vive Trackers
- Fix devices registration/unregistration. There is a bug: two trackers can be binded to the same Special_ID after turning tracker off and on (fix: save last free Special_ID instead of using NumTrackersMapped)
- We need some other format of data from fingers (because of thumb finger with its complicated movements). In a perfect world, Epics would have unified interface for fingers input from all devices (Touch, Knuckles, Leap Motion, VR gloves).
You can sample the skeletal data directly for the fingers and get per bone parent relative transform / component space transforms. The Curl/Splay values are just more generic methods of getting a basic movement. I was running gesture detection off of the full transform prior to the curl values being added in, but the Vive trackers are approximating the thumb motion anyway so until more granular inputs are available its not that useful.
Also I heard that Epic IS looking into a unified interface for that across Oculus/SteamVR for skeletal systems.