I have played CPP games Project Arena today and they are using UE4 to make the game, they have done a mixed reality trailer.
It would be great if they would tell us how they did it.
Interesting.
It does appear to be rather laggy in their version though it seems.
I might just have to go with using a Kinect and background removal and dropping that as a sprite in the game or something at this rate.
Also of interesting note I can see that the Oculus Touch controllers seem to lose tracking a few times in that demo which is not promising. The layout of the controller does seem to better suit that sort of game but hopefully the tracking is as good as the vive.
Kinect can work well, here’s some mixed reality stuff I’ve done with Kinect v2 and UE4:
That is really good!
What is the process for doing that?
How do you feed the masked image into UE4 ?
I’m using network mode for the third person mirror window.
Just using the open source UE4 kinect plugin; I posted some code to the forum post about how to get it running on 4.11.
hah love the dog tracking 1 mate!
Which plugin 1 are you using? Not the Kinect 4 Unreal 1 im assuming if you are talking about open source 1?
Ah yep checked your posts. Thought Lion abandoned that 1 though a long time ago?
Couldn’t find your specific details about the mixed reality side of things there though unfortunately. Any chance of a link or demo project?
I couldn’t find an answer hub post about these so I’ve made one:
I’d love to hear Epic’s official solution.
Great idea, but the only issue I see with the Kinect is the bandwidth requirements. I have a fairly high end rig, x99 deluxe motherboard and ecannot run the Vive and kinect at the same time due to bandwidth constraints on the USB 3.0 ports. Had to order an innatec usb 3.0 pci express card for the kinect.
I have been having the same problem trying to run an Xbox One controller with the vive and 360 controller attached too, regardless of which ports I use for some reason, really annoying. Haven’t got any problems with the Kinect and Vive running on separate USB busses though so you should be fine with the PCI 1.
So is it possible to output two views to the monitor yet or is this a unity only thing?
Id like to display a camera view next to the player hmd view but it’s looking like this isn’t possible.
That’s the main question. Realistically we need something like the Steam VR Unity plugin that outputs multiple layers, so that foreground and background can easily be composited later. A good explanation of the technique is here:
I’ve been looking into how to do this with what we have at the moment and it looks like the easiest option is to record the orientation / position of the controllers and the camera and then replay that. Has anyone else tried this approach? I’m planning to shoot the trailer on a sound stage / green screen hand held. What’s the results of doing match moving using something like Nuke or 3DEqualizer vs using the controller to track the camera? I assume the advantage of match moving is that you get the correct camera parameters (zoom / FOV) and that they can change during a shot.
We record the headset and controller transforms as well as the input buttons every frame. Then save that out to a file. We can then replay the interaction after the fact.
We have used it to generate 360 stereo video, but it would work fine for being able to see the playback overlaid on video if you had a camera setup to do it.
What’s your workflow for 360 stereo video? Did you use the Stereo Panoramic Movie Capture plugin?
We did use the Stereo plugin as a basis though needed to modify it for our specifics since we allowed the camera to move during the capture. We needed to do some stuff to keep the physics from getting out of whack. We also integrated our headset and hand position recording to this.
Handy way to get it out but then you can’t get the proper layering with objects in front of the person and also can’t do a realtime view in game with that method.
Forwarding to Fresher Thread : In case somebody stumbles on this thread, update in this direction : Mixed Reality Status update - VR and AR Development - Unreal Engine Forums