VR Expansion Plugin

[mordentral]

@ctpeepee

Hi ,

Got a few questions about your OpenVRInput module and the official SteamVRInput plugin. I don’t fully understand how some of stuff works so apologies if some of my questions don’t make sense.

• Firstly, am I right in thinking the documentation you mentioned hasn’t been updated yet regarding replication?
• Is there any reason to use the official plugin over yours? You say you have made it cleanly handle using official plugin but still allowing your gesture detection, why use the official plugin at all though as yours can also handle the skeltal input?
• Can SteamVRInput/your plugin handle using a vr glove controller (eg manus)? might not really make much sense as, at least for the steamvrinput plugin, i don’t think thats the point of the plugin. But for your plugin at least if it was possible then the replication/gesture detection would be very useful.

Thanks!

1. Yeah I got sidetracked dealing with some things with it and haven’t updated the documentation yet. Generally its just ticking on “replicate skeletal data” and choosing a replication mode. There are no real complicated steps. I’ll get around to updating the documentation soon, I only finished final testing last night and just need to clean some things up now.

2. Mine only handles the action system currently because the built in engine version does not contain skeletal input capacity yet and I didn’t want to ship a module that “requires” another party’s module to function. The Valve plugin also handles the action system and when it is installed mine turns that part of itself off in order to let it work as intended, basically it comes down to if you prefer the valve plugin or the native engine “Beta Input” style (mine is the beta input but with modifications to allow the skeletal system). Eventually when the engine gets some patches in that fixes the problem i’ll be removing that part of my module entirely.

As far as what the difference is from the valve plugin with that aside, theirs doesn’t handle mapping the hands to full body meshes, or replication, or easy gesture detection. They also are using some proxy inputs and a large quantity of input keys for each supported controller for mapping inputs which is a bit of a mess and some may prefer to not deal with until it is re-factored. If you are doing single hand meshes in singleplayer though I don’t see a reason why you couldn’t / wouldn’t just use theirs.

3. If the vr glove controller makes OpenInput drivers for them then they would work out of the box, since they use vive trackers in their promotional material I would hope that they do it correctly and actually write a vendor driver for them for SteamVR and everything “just works”.

[ctpeepee]

1. Yeah I got sidetracked dealing with some things with it and haven’t updated the documentation yet. Generally its just ticking on “replicate skeletal data” and choosing a replication mode. There are no real complicated steps. I’ll get around to updating the documentation soon, I only finished final testing last night and just need to clean some things up now.

2. Mine only handles the action system currently because the built in engine version does not contain skeletal input capacity yet and I didn’t want to ship a module that “requires” another party’s module to function. The Valve plugin also handles the action system and when it is installed mine turns that part of itself off in order to let it work as intended, basically it comes down to if you prefer the valve plugin or the native engine “Beta Input” style (mine is the beta input but with modifications to allow the skeletal system). Eventually when the engine gets some patches in that fixes the problem i’ll be removing that part of my module entirely.

As far as what the difference is from the valve plugin with that aside, theirs doesn’t handle mapping the hands to full body meshes, or replication, or easy gesture detection. They also are using some proxy inputs and a large quantity of input keys for each supported controller for mapping inputs which is a bit of a mess and some may prefer to not deal with until it is re-factored. If you are doing single hand meshes in singleplayer though I don’t see a reason why you couldn’t / wouldn’t just use theirs.

3. If the vr glove controller makes OpenInput drivers for them then they would work out of the box, since they use vive trackers in their promotional material I would hope that they do it correctly and actually write a vendor driver for them for SteamVR and everything “just works”.

1. Sweet. Just checking I wasn’t being blind.

2. Thanks for all the info! I don’t really understand some of what you’ve said but as I work a bit more with stuff I’ll check back and hopefully the other things will make more sense.

3. Unfortunately that is not the case. The gloves work completely separately to steamvr and the trackers are independently used to position the wrist/hand. I’m gonna submit a support ticket and ask if they have plans to do that now I know though!

My project is multiplayer and will have full body ik so from what you’ve said looks like your plugin might be very helpful. Does your gesture detection/replication work only for the skeletal data from your open input module or will it also work if skeletal data coming from SteamVRInput or from something like the manus glove? Main focus for me is actually manus and then supporting vive controllers as a fallback.

[mordentral]

@ctpeepee

1. Sweet. Just checking I wasn’t being blind.

2. Thanks for all the info! I don’t really understand some of what you’ve said but as I work a bit more with stuff I’ll check back and hopefully the other things will make more sense.

3. Unfortunately that is not the case. The gloves work completely separately to steamvr and the trackers are independently used to position the wrist/hand. I’m gonna submit a support ticket and ask if they have plans to do that now I know though!

My project is multiplayer and will have full body ik so from what you’ve said looks like your plugin might be very helpful. Does your gesture detection/replication work only for the skeletal data from your open input module or will it also work if skeletal data coming from SteamVRInput or from something like the manus glove? Main focus for me is actually manus and then supporting vive controllers as a fallback.

The skeletal replication currently IS of the OpenInput skeletal data, I haven’t expanded it to cover anything else yet. Assuming they correctly create their SteamVR controller driver they should be able to pass out the skeletal pose through the API and have it just work.

[Simply_Clueless]

Hey man! After getting your advice on using the slot grips I found the tutorial you had on the documentation website and it seems to be working alright yet my objects scaling is being multiplied by 100. In the actual object which is a disc, I have the scale as 0.1 but when I grab it it sets it to 10. Any idea on how to fix ?

[mordentral]

@Simply_Clueless

Hey man! After getting your advice on using the slot grips I found the tutorial you had on the documentation website and it seems to be working alright yet my objects scaling is being multiplied by 100. In the actual object which is a disc, I have the scale as 0.1 but when I grab it it sets it to 10. Any idea on how to fix ?

When you get a socket world transform it has its parent object scale, so when you make it relative to the object it scales incorrectly (doubles up the scale), when you are passing out the transform set the scale to 1. I may be switching over to that node passing out the relative space transform instead soon though, depends on how that would effect users as generally just passing out world transform of things is easier.

Edit That is assuming that you are overriding that function, if you aren’t then it should be working correctly.

[Simply_Clueless]

I haven’t overridden that function just yet. I’ll have a look later when i’m at home but I feel like it should be correct judging by the images on your website!

[Simply_Clueless]

Here is some screen shots on how I have done it which should be pretty identical to how you showed.

[mordentral]

@Simply_Clueless

Here is some screen shots on how I have done it which should be pretty identical to how you showed.

Well for one you aren’t casting or checking if the object has the interface first before checking DenyGripping on it, so the result out of that message is invalid (there may be no interface to get a correct result out of for that object). I assume that your object is a GrippableActor so you should be passing the actor reference around not the component (checking for interface between the two would have handled that).

That aside, yeah, I actually had a mistake in my socketing blueprint example and had the pins backwards for the socket one. I fixed the tutorial and re-uploaded corrected images, I also made the snapping images and example be of an actor as usually you will be gripping actors. That one was my bad, I was rushing during that part of the tutorial trying to get the rest of the site up.

[DaBoss3D]

Hi,

Thank you for your awesome work on the plugin!

I just want to ask if someone had a issue like mine, that can point me to the right direction.

In Multiplayer, the client cant’t teleport (just see the short blue teleport line that almost immediately cuts off),
but the strange thing is that the player on the server side see the Teleport location of the client in full (with arc and target).

The levels have NavMeshBounds.

I’m pretty new to UE but not c++

Thank,
Carlos

[ctpeepee]

Set up your OpenInput plugin today. Everything worked great up until I tried enabling replication. When run in engine with use single process disabled, the client (non-vr) crashes either on startup or shortly after. Had two slighlty different error messages (below). I tried messing around with the replication settings to see if they made any difference, found that the only thing that stopped it crashing was to disable Smooth Replicated Skeletal Data but that with that enabled there was no replication. Any ideas?

Assertion failed: (Index >= 0) & (Index < ArrayNum) [File:C:\Program Files\Epic Games\UE_4.22\Engine\Source\Runtime\Core\Public\Containers/Array.h] [Line: 611]
...
UE4Editor_OpenInputPlugin!DispatchCheckVerify<void,<lambda_ac839d0d620246989338f24bfe6f099f> >() [c:\program files\epic games\ue_4.22\engine\source\runtime\core\public\misc\assertionmacros.h:162]
UE4Editor_OpenInputPlugin!FBPSkeletalRepContainer::NetSerialize() [c:\users\...\plugins\openinputplugin\source\openinputplugin\public\openinputfunctionlibrary.h:455]

Assertion failed: (Index >= 0) & (Index < ArrayNum) [File:C:\Program Files\Epic Games\UE_4.22\Engine\Source\Runtime\Core\Public\Containers/Array.h] [Line: 611]
...
UE4Editor_OpenInputPlugin!DispatchCheckVerify<void,<lambda_772c629cba1e4a100b11566d43231efa> >() [c:\program files\epic games\ue_4.22\engine\source\runtime\core\public\misc\assertionmacros.h:162]
UE4Editor_OpenInputPlugin!UOpenInputSkeletalMeshComponent::FTransformLerpManager::UpdateManager() [c:\users\...\plugins\openinputplugin\source\openinputplugin\public\openinputskeletalmeshcomponent.h:279]
UE4Editor_OpenInputPlugin!UOpenInputSkeletalMeshComponent::TickComponent() [c:\users\...\plugins\openinputplugin\source\openinputplugin\private\openinputskeletalmeshcomponent.cpp:224]

[mordentral]

@ctpeepee

Set up your OpenInput plugin today. Everything worked great up until I tried enabling replication. When run in engine with use single process disabled, the client (non-vr) crashes either on startup or shortly after. Had two slighlty different error messages (below). I tried messing around with the replication settings to see if they made any difference, found that the only thing that stopped it crashing was to disable Smooth Replicated Skeletal Data but that with that enabled there was no replication. Any ideas?

Thanks for the report, I tracked it down and is was a missing safety check, it only shows up with delayed loading, the module is updated with a fix.

I’ll note btw, you don’t likely want to run as multiple process for testing , the engine doesn’t export non engine directory editor modules in -game running instances so the animation editor module won’t load. They had to go and specifically fix with launching from the uproject awhile back but didn’t fix case as well.

You’ll either want to package it entirely, or run testing as single process, its a universal issue with editor plugins with custom animation nodes AFAIK.

[ctpeepee]

Thanks for the report, I tracked it down and is was a missing safety check, it only shows up with delayed loading, the module is updated with a fix.

I’ll note btw, you don’t likely want to run as multiple process for testing , the engine doesn’t export non engine directory editor modules in -game running instances so the animation editor module won’t load. They had to go and specifically fix with launching from the uproject awhile back but didn’t fix case as well.

You’ll either want to package it entirely, or run testing as single process, its a universal issue with editor plugins with custom animation nodes AFAIK.

Thanks for the fix and for the tip! (Just tested, all working! Doesn’t crash when doing what I was doing before and replication all works great when testing it on single process.)

In general is there a way to test with single process with a vr and non vr player? I started using multiple process’ to test stuff that needed one player in vr because couldn’t figure out a way with single process.

[mordentral]

@ctpeepee

Thanks for the fix and for the tip! (Just tested, all working! Doesn’t crash when doing what I was doing before and replication all works great when testing it on single process.)

In general is there a way to test with single process with a vr and non vr player? I started using multiple process’ to test stuff that needed one player in vr because couldn’t figure out a way with single process.

While you can tick on UseDedicated with a vr player the second instance of another client still kills it in single process for a VR player, so no, not that I am aware of.

[Swinny]

You are using root motion movement with a VR Character?

Or do you mean you are using it with a VRCharacter but it is possessed by AI?

I removed the built in skeletal mesh from the component array as I had assumed that no-one would be using root motion for movement in VR and it would have to be re-parented anyway. I would be interested in (any) good reason why someone would actually be using it. I’ve actually been actively removing root motion checks in the character movement component in order to avoid the (minor) perf hit of checking if it is active.

I am using root motion in the player character - I am using ikinema in concert with your plugin to have true tracked presence in the world, the root motion allows for realistic movement when using thumbsticks - on that note however - how do you move the characters capsule when using VRcharacter by head movement? my current setup (not using the template - i know that works though) leaves the capsule. is there a simple bool im missing?0

In a perfect scenario i would have the player be able to move the character by VR movement in the playspace alone, with the skeletal mesh updating walk animations (not root motion) to display those small movements to the player - but when using the thumbsticks to move around, have the root motion animations handle everything.

Could you add a version of the VR character with the Skeletal Mesh component intact? That way current functionality would not be hindered but for those of us that want it, can use it?

Thanks for the reply

[mordentral]

@Swinny

I am using root motion in the player character - I am using ikinema in concert with your plugin to have true tracked presence in the world, the root motion allows for realistic movement when using thumbsticks - on that note however - how do you move the characters capsule when using VRcharacter by head movement? my current setup (not using the template - i know that works though) leaves the capsule. is there a simple bool im missing?0

In a perfect scenario i would have the player be able to move the character by VR movement in the playspace alone, with the skeletal mesh updating walk animations (not root motion) to display those small movements to the player - but when using the thumbsticks to move around, have the root motion animations handle everything.

Could you add a version of the VR character with the Skeletal Mesh component intact? That way current functionality would not be hindered but for those of us that want it, can use it?

Thanks for the reply

I could add the skeletal mesh back in but that wouldn’t fully fix root motion, it would need several things added back in that were removed as minor performance savings.

I also don’t understand how thumbsticks and root motion would work in concert like that…You still need head motion influence during walking which would break your “root motion” setup. And its possible to use a non linear movement without root motion as well.

The SimpleVRCharacter would be closer to what you want, it does not remove the root motion features, but it does move the entire character with the HMD motion.

Or you could run a character and a proxy actor for the HMD containing and sync the two.

Edit That being said, I have been considering sitting the skeletal mesh back in and re-parenting it.

[jawatr]

Hi ,

Quick question: can the VRExpansion Plugin work on Oculus Quest? Is it dependant on Win64 architecture or SteamVR to work? I just want to know if it´s possible at all before attempting what it looks like a very painful process to port my game there

Thanks!

[mordentral]

@jawatr

Hi ,

Quick question: can the VRExpansion Plugin work on Oculus Quest? Is it dependant on Win64 architecture or SteamVR to work? I just want to know if it´s possible at all before attempting what it looks like a very painful process to port my game there

Thanks!

Yup, I packaged up the example template last night to test and after enabling multires and switching it to static lighting only it did just fine with all features intact. Obviously you would have a road ahead of you downscaling to fit the hardware but that is a given.

It is my understanding though that they don’t just let any and all games ship to quest and are gating new quest titles behind pre-approval.

[jawatr]

Yup, I packaged up the example template last night to test and after enabling multires and switching it to static lighting only it did just fine with all features intact. Obviously you would have a road ahead of you downscaling to fit the hardware but that is a given.

It is my understanding though that they don’t just let any and all games ship to quest and are gating new quest titles behind pre-approval.

That’s great! Good to know there is light at the end of the tunel

Btw, I just tried to package for the Quest but I’m getting error:

LogPlayLevel: [1/4] Module.VRExpansionPlugin.gen.2_of_10.cpp [armv7-es2]
LogPlayLevel: In file included from E:/UE4/GladiusQuest/Plugins/VRExpansionPlugin/VRExpansionPlugin/Intermediate/Build/Android/UE4/Development/VRExpansionPlugin/Module.VRExpansionPlugin.gen.2_of_10.cpp:1:
LogPlayLevel: In file included from E:/UE4/GladiusQuest/Intermediate/Build/Android/GladiusQuest/Development/Slate/SharedPCH.Slate.h:73:
LogPlayLevel: In file included from C:\Program Files\Epic Games\UE_4.22\Engine\Source\Runtime\Slate\Public\SlateSharedPCH.h:231:
LogPlayLevel: In file included from C:/Program Files/Epic Games/UE_4.22/Engine/Source/Runtime/CoreUObject/Public\UObject/Object.h:12:
LogPlayLevel: Error: C:/Program Files/Epic Games/UE_4.22/Engine/Source/Runtime/CoreUObject/Public\UObject/UObjectBaseUtility.h(483,14): error: incomplete type ‘USkeletalMeshComponent’ named in nested name specifier
LogPlayLevel: return IsA(T::StaticClass());

I have same error 4 times, do you know what’s happening?

Thanks again for your help

[ctpeepee]

Hello again,

I noticed the other day that a static mesh component attached to a grip motion controller comp didn’t appear to be getting late updates and so floats behind the controller. It wasn’t particularly important so wasn’t gonna ask and just hope I figure it out at some point in the future but after messing around with it more now curiosity has got the better of me! Why is the case?

I read about the GatherLateUpdateComponenets function and can’t see a reason why it’s picking up all the other components except the static mesh. I also tried manually adding the mesh to the *AdditionalLateUpdateComponenets *but presumably whatever reason the static meshs aren’t late updating anyway also means that doesn’t do anything.

[mordentral]

@jawatr

That’s great! Good to know there is light at the end of the tunel

Btw, I just tried to package for the Quest but I’m getting error:

LogPlayLevel: [1/4] Module.VRExpansionPlugin.gen.2_of_10.cpp [armv7-es2]
LogPlayLevel: In file included from E:/UE4/GladiusQuest/Plugins/VRExpansionPlugin/VRExpansionPlugin/Intermediate/Build/Android/UE4/Development/VRExpansionPlugin/Module.VRExpansionPlugin.gen.2_of_10.cpp:1:
LogPlayLevel: In file included from E:/UE4/GladiusQuest/Intermediate/Build/Android/GladiusQuest/Development/Slate/SharedPCH.Slate.h:73:
LogPlayLevel: In file included from C:\Program Files\Epic Games\UE_4.22\Engine\Source\Runtime\Slate\Public\SlateSharedPCH.h:231:
LogPlayLevel: In file included from C:/Program Files/Epic Games/UE_4.22/Engine/Source/Runtime/CoreUObject/Public\UObject/Object.h:12:
LogPlayLevel: Error: C:/Program Files/Epic Games/UE_4.22/Engine/Source/Runtime/CoreUObject/Public\UObject/UObjectBaseUtility.h(483,14): error: incomplete type ‘USkeletalMeshComponent’ named in nested name specifier
LogPlayLevel: return IsA(T::StaticClass());

I have same error 4 times, do you know what’s happening?

Thanks again for your help

No I don’t, that is an issue with the generated source and kind of out of my control, it also isn’t happening for me with a clean template. You aren’t trying to nativize are you?