VR Expansion Plugin

Development Platform & Builds XR Development
1

VR Expansion Plugin

Updated: Constantly

EpicMegaGrants_Badge_Dark_Small.png
EpicMegaGrants_Badge_Dark_Small.png528×546 24.7 KB
](filedata/fetch?id=1697558&d=1576159007)

https://i.imgur.com/cFdzy5E.png

Playable Template Demo
https://drive.google.com/file/d/1TUF…ew?usp=sharing

Plugin Website:
https://vreue4.com/

Plugin Partial Features List:
https://vreue4.com/features

Documentation
https://vreue4.com/documentation

Website Hosted Patch Notes
https://vreue4.com/patch-notes

plugin is intended to ease the creation of VR games/experiences in UE4. It is up to quite a few different additions now and has a fleshed out wiki to help ease into it (may be a version or two behind at times, I attempt to update it when I can, the template is generally the best informational tool). It is also intended to be compatible with multiplayer experiences without any direct changes from the end user, most of the relevant components / actors replicated themselves over the network with some customization included.

It is fully Blueprint implemented but can be manually linked to and used in c++ if needed.

Consider Supporting Me On Patreon

Some Games In progress or released that currently use the plugin (These are just ones I know about)
https://vreue4.com/games

Plugin Pre-built Downloads
https://vreue4.com/binaries

Plugin Repository
https://github.com//VRExpansionPlugin

Plugin Example Template Repository
https://github.com//VRExpPluginExample

Repository Installation articles By Community Member kusogaki77

Installation

Tips & Tricks

Template Note:

I would like to note by the way that everything currently in the template is an example of what I think is best practices (though maybe not implemented as well as I would for an actual game), not everyone is going to need the same set up and not everyone is going to need a full VRcharacter. Custom pawns that just use the motion controllers and camera / parent relative component would be useful to people that don’t want gravity / collision. It also has to give examples of far more things than an average game is likely to use, so there WILL be refactoring involved if it is used as a base for one (input bindings for one).

7 Likes
2

Fantastic! That replication is going to make life so much easier for me.
Gonna have to download it and give it a try!

3

Awesome work, looks great!

  • If I wanted to use Mitch’s template for the pickup system, do you think I’d be able to just replace the Attach calls with your Grip function (e.g., would replication still work there?)? I found the system in the SteamVR template that uses the GripMC to be a little cryptic in terms of being able to easily extend it and implement different sorts of items.

As an aside, I still think it would be beneficial to change how the Lock to HMD functionality works so that it would move the actor itself, rather than just components attached to the camera. I looked into it and it seems like that could be done by just recalculating the camera’s matrix relative to the new actor matrix and then setting it on the actor where it currently does the component-move.

I still think only moving the components could eventually cause gameplay problems (especially in cases of larger levels, multiplayer etc.), and have seen a few posts to that effect.

4

Yeah it shouldn’t be a problem, you’d have to keep in mind that Mitch’s examples use attached simulating components with Damping to keep them relative to the controller (if interactive), plugin uses the sweeping functions to move items manually relative to the controller instead (and sends their components to the late update function). So there would have to be “some” changes, the plugin forces simulation off when gripping an actor so a bunch of nodes could / should be skipped from his example if using the grip function. I was considering implementing AttachTo as well as an option but after playing with it I didn’t like how it worked so I scrapped my plans to support it.

In my demo project I run all of the “Character has in his left/right hand, drop if he picks up that, ect ect” logic in BP and just call Grip/Drop in the plugin on what I decided to do from the server. It is game logic agnostic and just handles holding and dropping and triggering OnHit events.

The steamVR template is very WIP and I get the idea that proteus is lacking in free time at the moment. The basics of gripping an object with my plugin is fairly simple, there is a write up on the motion controller wiki page with the basics and some best practices. I would have used Epics “World/Relative/Snap” enum like in attachment but passing in actual transforms has more total flexibility.

wiki/VRGrippableMotionController

Note Attachment is already replicated by default in the engine, when you attach a component it turns off replicating movement from the server and the clients handle it from then on out (until detached). That is one of the reasons why it doesn’t sweep hit events on attached actors. You don’t actually have to use the grip functions at all if you don’t want to, they are just my preferred way of doing it.

I may add back in having the server manually sweep actors attached to the controllers so that OnHit events still trigger, it was originally something I had it doing but removed it when I went over to only using the grip functions.

It wouldn’t be that difficult to create a custom pawn that samples the HMD position instead of the camera and offsets its location by the difference from the last sample point.

The thing is that OpenVR returns the HMD location in relative space to the tracking volume you have set up so using the camera in relative space to the chaperone in the engine is a very clean solution to it as the pawn can act as the tracking space (that you move with teleport / other movement inputs) and the components as the relative location within that tracking space, you are essentially moving around the tracking volume with you.

Do you have any quotes/links about the concerns? As I said it wouldn’t be hard to implement I just don’t see the point in doing so, but I would like to read about any issues that the current method presents as if they are significant enough I would have to reconsider LockToHMD. I haven’t been able to think of anything that would be a problem for me so far and also haven’t run into any issues with the default setup. You aren’t moving component farther from the pawn itself than the size of the tracking volume in the default setup.

5

I’m looking into smoothing replication at lower update rates by the way. I’ll need to test of bunch of different rates and try some lerp/prediction tests to see what feels best and where it actually becomes a problem first. The default 100htz (matching pawn default) rate didn’t need any smoothing that I could see so I left it out to get it out the door but I could see needing to add it in for 30htz or so (though it is only a per client network load and isn’t that heavy).

6

Hey . Thanks for putting in the effort to make awesome plugin! It really helps with jumping straight into VR and building stuff. I’ve been using your VRGrip plugin for about a month now. It helped a lot, but I have run into some issues.

Migrating to VR Expansion Plugin
I’m interested in migrating from your old plugin to one, but I’m seeing some issues trying to rip out the old one and replace it with one. I wonder if there’s something simple I’m missing. Depending on how many people used your other plugin, it might be helpful to add a How-To to the readme.

Collision Event Bug.
I have a SphereCollision component parented under the grip controller. I’m using the begin/end overlap events on collision to determine which object I can pick up. I’m noticing that when the grip controller is holding an object then BOTH overlap begin and end events are triggered on every tick. The events are only triggered on the colliders close to the object, but they are triggered on every tick regardless of which controller is holding the object. That is to say, if I hold an object in my right hand, then my right collider is triggered on every tick. Then if I bring my left hand close enough to my right hand, that collider is also being triggered on ever tick - even though the right hand is holding the object.

rapid-fire of begin/end overlap events is causing incorrect results when I poll which objects I can pickup. If I had to guess, I’d say something about the way the plugin is updating the gripped object position is causing collision events to freak out. I noticed the wiki suggests that I use a sphere trace instead of overlap events - probably for reason. I’ll use the sphere trace for now, but performing a sphere trace every tick is probably more expensive than using overlap events.

Better Physics Behavior
Have you read on how they solved physics-based object movement in unity? It’s a really interesting read, and they put their work up as a

Obviously their solution won’t be ideal for everyone, but I do think they address a number of areas that the sweep-to method falls short in. Namely…

Objects get caught on floor easily.

What do you think about trying to port some of their ideas to UE4?

1 Like
7

Migration
Being a new plugin the classes technically are considered separate from their old versions, you actually have to place them back into the pawn and re-place their functions. I decided to rename it as the old name no longer suited the project but is a sad consequence being a one time hit of placing the nodes again.

Overlap Triggering
I set move ignore to the controller when gripping an object, it is possible that overlap is triggering anyway as the current movement is swept location. You can either not use overlaps or set it to ignore the channel you place the overlap on after gripping (I have a special collision channel for traces / overlaps). I also don’t call traces every frame, I call it on button press, but if you are looking to highlight objects to pick up I see your point. But you can always run traces on a timer set to 30-60 htz for highlighting and call the trace on button press for the actual pick up (my voxel project does for block highlights).

Push / Pull
Yes you actually can, the object has to be simulating though for it to move it, i’ll agree that the interaction is less than steller currently and I am going to provide a better option for real physics. The current grip is intended just for when you don’t need full physics solving on the gripped object itself as it will be cheaper. It was also the fastest one out of the gate to implement so I could look at other things.

New Grip Styles
I fully intend to make a better grip option using physics, I originally had AttachTo set up and ran into it not being up to what I wanted from it and removed it from the grip function. Unity’s solution is effectually what I am currently doing with location but done through the physics system instead so it actually slots right in. It is next on my Trello list actually to look into it now that I have decided to throw away AttachTo (spent my free time in the last day mulling over if AttachTo was worth it and trying it out).

I don’t like the visual drag latency in how they are doing it in Unity though and am pretty sure that there is a way around it by extending my current solution and using constraints (ripping from grab handles but forcing late updates and removing the lerp). I really like the interaction in say Hand Guns, Horseshoes, and Hand Grenades but don’t like that the guns themselves lag behind hand movements visually (for actual shooting location they are correct so there is something to be said for that).

TL;DR yes all of those things bug me as well and as a user of my own plugin I am fixing them. I’ll move experiments to the side for now (was playing with movement options) and just implement the new grip option instead.

The cool thing about the plugin is when the new grip is in you’ll just have to select it from the grip style drop down and it should work.

Edit Oh and I was looking into both hands on the same object interactions. Eventually both options should support that as well; but I will wayside it until the new grip is in too.

8

I actually had done yesterday but got side tracked trying to make a hybrid system (not sure how stable I can get that one yet). It needs some going over to make sure that multiplayer works fine with the new grip style before I merge it in but I think the video is pretty much what you wanted.

Its up to 5 grip methods now I think (6 with hybrid if it makes it), you pick what one you want per item you are picking up and how you want it to behave. When it comes out method will have several configurable options (you probably don’t notice in the video but I have the grip strength 1000x higher on my right hand than my left so it doesn’t free float as much in the video).

I’ll get it pushed out to the main branch of the plugin by the end of the weekend, I was off on a tangent for awhile there experimenting with a bunch of random things (VR gets me excited) or would have already been in.

Cheers

9

Thanks for the help. As it turns out, I had to delete the original VRGrip components before deleting the plugin, or my game would crash on load. After deleting the components, it was okay if I still had any variables with the VRGrip type (they got reset to Object). Then I just had to put the new plugin in place and run around reconnecting everything (that was less fun). Next time around, I think I’ll integrate the plugin with my game via C++ so it’s easier to refactor things. Blueprints makes refactors surprisingly hard with it’s propensity to disconnect nodes! And speaking of C++, I don’t know if you intended the plugin to be using in C++ or not. I’m definitely a noob at both C++ and Unreal, but it looks like the plugin classes are missing the VREXPANSIONPLUGIN_API (macro? keyword?) in order to be exposed to game logic. I can send a pull request that adds if you’re interested.

You’re correct, I’m using it for highlighting. I forgot to mention that in my original post. I’ll have to play around with running my traces on a slower frequency, that’s a good idea! I’m a little worried that there will be a disconnect between the feedback the user sees and the action that occurs - but I won’t know how bad it is without testing.

I totally agree. I think having really good physics interaction is worth the trade off of having some latency on gripped objects, but it’s definitely not a compromise I’d like to make if I don’t have to.

Oh man, looks super awesome!! Yeah, it looks like most of the things I was interested in. I can’t wait!

10

Yeah I forgot the macro on the classes, I’ll add it in to them today.

11

Merged the physics grip branch into main, it needs more play time to lock down bugs but was working for me during initial multiplayer testing (don’t forget to replicate movement on physics objects so they stay sync’d). The hybrid setup “works” but needs to be finished still.

InteractivePhysicsGrip is the style you were looking for, it also adds two properties to the motion controllers for Stiffness and Damping when holding physics simulating objects, I may break that out into a object by object basis later.

There are also a few new functions and features, i’ll document them tomorrow.

12

Would any of the exposed OpenVR functions be able to return what kind of HMD is in use? Rift/Vive/OSVR. I need to change player class since the haptic events are very different on Vive vs Rift.

13

I think someone requested of to have his library include that function. It is already in GEngine so I will throw it in anyway since it makes sense and is easy, but if you don’t intend to use the rest of my plugin and already use’s library then you should be able to find it there.

Regardless I’ll have it in by the end of the day (have to push out some cleanup for things anyway later).

Edit Pushed the new function early as the rest of my work today will be on a new branch

GetHMDType - static

15

Oh my, Plugin couldn’t have come at a better time. Literally just spent last weekend testing my Vive game with some friends, and one of the biggest issues was the grabbing system I had made. My solution, using merely the built in blueprint functions was to turn off gravity on grabbed components. wasn’t the ideal solution but it worked for the most part. That said it seems plugin gives far better control and I hope to implement it.

Could you by chance give some insight into the setup of the “New grip style” video? I’ve got a line trace on the controller trigger, then checking my grabbed object and then calling grip actor from your plugin, but it seems to go a bit crazy once attached. Will have a further tinker, but does look promising. Great work!

16

Thank you so much for the HMD type! makes my life so much easier. However, ParentRelativeAttachmentComponent does not seem to work at all. I looked at the code and it seems like it should but all that’s happening is occasionally, there will be an offset of the child component of the ParentRelativeAttachmentComponent. It just sticks at an odd rotation, and keeps following the HMD rot at that offset rotation 1 to 1. I tried with both Static and Skeletal meshes, and tried yaw values between 1-3 as well as much higher, locked on pitch and roll.

17

If it is “going crazy” after pickup and you are using the Interactive Physics Grip then you probably have collision on the controllers themselves enabled and it is overlapping. You’ll want to put them on their own channel and set the gripped object to ignore collision with that channel when held. The IPG uses a kinematic body and a joint attached to the gripped object, so it is still fully simulating and will collide with all other physics bodies that aren’t set to no-collide with it.

My test project has collision off on the controllers, but I used have it enabled and run them on the same channel as my traces and set held objects to ignore them on grip.

Do you have it set to 0,0,0 of the Camera? I use it for the base point of my IK arms and haven’t had a problem. Although I did have a commit up for all of 10 minutes with bAbsoluteRotation=1 in the attachment component by accident that I removed so I guess it is remotely possible you downloaded it in that 10 minute span?

Edit

Also please ignore the VRRootComponent in the files currently, I am iterating over several methods of having a dynamic root capsule and the first attempt was way too CPU heavy. I branched it to not effect default for now and am going to try re-writing all of the movement, penetration, sweep, and physics body transform update code in a primitive component instead.

18

Ah thanks for that. At first turned off collision on the controller meshes, but it seemed to still move around. Then changing the grip collision type to “Interactive Collision with Sweep” seemed to fix it, and now it works great. My only question is, would it be possible to add instead of actor to grip, component to grip.

In my game at the moment I’ve got a lot of blueprints with nested components with physics simulated. Hence I would trace for components rather than actors. Would be possible?

19

Interactive Collision with sweep won’t have true physics interaction by the way, it is there for cases when you don’t need true physics interactivity with the environment (will still interact with simulating objects) but still want held objects to not pass through walls. If you are just grabbing and moving things like handles then you would be better off using PhysicsOnly as it is faster and locked to the hand. Each grip method has its uses really, the non interactive ones are good for things like swords and items that you don’t want pulled from the hand on collision.

How much was the physics interaction grip moving for you though? I have it using late updates right up until collision with an object, then it will stop late updates to have less visual penetration. You can adjust the grip strength and physics damping in the motion controller properties currently, if the stiffness was too high and it was jittering when colliding with something then you can adjust the stiffness and damping to smooth it out. Also if the object being lifted has a lot of mass then you may need more stiffness to lift it without movement.

I intend to add component grip as an alternative to actor grip, it won’t be hard to add, literally just don’t gather the root component and use the passed in component instead. It would make things like levers and slides and things far easier to interact with as well.

I was planning on adding component grip at the same time that I move the Grip strength and damping out of the controller and into each grip call on a case by case basis.

Edit

I will mention that for multiplayer environments I’m not sure of the cleanest way to handle what you are doing. Setting movement replication on/off is Actor wide, not on a per component basis, so to be safe I will be turning off movement replication on the entire actor of a gripped Physics simulating component.

20

Just tested “Interactive Collision with physics” again now and it seems to be fine. I think what I forgot to do was to turn off the trace once it had hit an object. Right now my blueprint is a bit of a mess, since I’m trying to work from a duplicate, so things got a bit confused. Basically it was moving around a lot. I ended up analysing the VR template and found I had setup the World offset value slightly wrong as well, so I think that didn’t help either.

Good to hear about the grab component, for me that would be a greatly appreciated feature :slight_smile:

21

I’m actually adding it in now, needed a break from the VRRoot dilemma. Keep in mind that I am unsure how well nested simulating components are going to work with multiplayer, for single player it should be fine right off the bat but having them all on the same actor means that turning off movement replication turns it off for all of them.