[FREE PLUGIN] [VELOX PLAYER] Import and Play Volumetric Video from iOS devices with LiDAR

Velox XR introduces tools of Volumetric Video workflow from mobile devices into Unreal Engine 5.

Velox Player Plus is a code open-source plug-in that imports volumetric video files in the VLX format recorded on mobile devices with depth cameras. The VLX format allows creators to bring photorealistic appearance of people and environments into Unreal Engine scenes. The video frames are turned into animated geometry with textures which is fully compatible with the real-time rendering pipeline of Unreal Engine. With added benefits of the camera tracking and elimination of the background it makes a perfect tool to virtual production and gaming studios for creating immersive experiences based on real-life performances.

In order to record a 3D video you need an app installed on a device with a depth camera. There is an official iOS app “V3L Camera ‘’ available in the App Store for FREE. The process is fully mobile and requires:

  • NO green screen
  • NO special studio
  • NO camera calibration

This is a summary of all Velox Tools for Volumetric Video workflow

  • Velox Player Plus - an extension plugin to edit and save VLX files with generating masks and object detection

  • Velox Neuro - a FREE utility plugin to run ML inference in UE

The Volumetric Video workflow has 6 simple steps:

Watch this example of a video created with Velox Player tools in Unreal engine 5

Also learn this tutorial before get started:

Also watch this workflow with the Velox Player Editor from the Plus Plugin

Documentation links:

Sample UE5 Projects:

2 Likes

This looks amazing! Awesome efforts!
I wonder how I can apply the mask from texture (not video, nor live feed).

Hi. Thanks for the question. It’s a nice feature. We will add that in one of the next updates. Although there is a workaround how you can replace the generated masks with your ones.

If you have the Velox Player Plus plugin all your generated masks are always placed into your local temp folder for cache before saved into your VLX file.

The temp folder location can depend on your platform and OS version but for Windows you can find it here: C:\Users<your user name>\AppData\Local\Temp\VeloxPlayer then find a folder starting with the name of your VLX file + .tempdir:
For example: Dataset2023_09_29_12_39_57.tempdir

There you can find the mask folder where you can replace the files with your version. Each file corresponds to one frame.

Hope that helps.

1 Like

Thank you for your quick reply, much appreciated!
I would like to ask another question, I’ve downloaded the demo and really liked the virtual background remover from webcam, results looks great!
My second question is, instead of having the effect (background removal) applied from the live feed of the webcam (or from video), how can i make it apply from a material instead?
And thanks again for your reply.

Sorry for my late reply. In fact the background removal is done by applying a mask texture in a material already. A mask texture is in the Grey format from 0 to 1 which you can use as alpha transparency.
If you look at the NeuroSample there are two Blueprints:

  • DisplayMask - function OnNewTexture
  • DataPlayer - function OnInitMask
    where it shows how mask texture is obtained and applied into the material.
    Hope that helps.

*** UPDATE ***

Velox Player plugin version 0.7
Velox Player Plus plugin version 0.4b

VELOX RECORDER

Present your game play and content directly from the game!

Create a video of yourself walking through the live action and environment in 3D
Velox XR is excited to introduce a new feature for recording VLX videos of a live gameplay.
In the Velox Player Plus plugin we added a new Blueprint Actor: Velox Recorder which can be placed into a scene to automatically record a gameplay footage of a Player from the Recorder’s camera view.
This is a runtime feature which can be built into a standalone UE application running on any UE compatible platform including VR.

By combining the game VLX footage with another VLX video of real people you can produce a never-seen-before experience of putting reallife performances into live 3D games by just using your mobile device:

  • no green screen is required
  • no studio is required
  • no hardware calibrarion is required

Install Velox plugins from Epic Marketplace now and start filming with V3LCamera iOS app from App Store.

Example Video

Tutorial Video

Documentation

UPDATE

Velox Player Pro: version 0.5 is available on Epic Marketplace

We removed dependency on the Velox Player Plugin. The old Velox Player will stay in its current state as a free player.
The new Velox Player Plus is a brand new standalone plugin with lots of new quality and performance improvements:

Top features
:1st_place_medal: Unreal Engine 5.4
:2nd_place_medal: iOS / Android / VR
:3rd_place_medal: Relight with AI normal maps

New
:arrow_right: Mesh animation with faster rendering
:white_check_mark: Materials: Basic, Color Correction, Chromakey
:arrow_right: Runtime filters for depth correction
:white_check_mark: Pipeline UI
:arrow_right: ROI track Actor
:white_check_mark: VeloxModBox Culling
:arrow_right: Recorder

Check out the new Tutorial Video
New Documentation
New UE Demo project