The source code (Unreal Engine 4.7) for my “Breath Tech Demo” is now available for free download. It demonstrates how to detect the players breath using a microphone and show it in VR with a direction and velocity to allow interaction.
I’ve included some rough notes below on the various approaches I investigated. I’m new to game development so don’t assume these are the best approaches, but hopefully they may save someone a little time or inspire other new developers to experiment.
Please note, I’m very busy working on my own project so can’t offer any support with the code.
Sampling
Samples captured by a small C++ routine using UE4 voice capture API. API details found at quoteunquotestudio.com - thanks for sharing. I’m using the minimum 8Hz sample rate. Nothing more is required for breath detection. Check for new samples every tick. I calculate the amplitude being reached by at least 40% of the sample, compare it to the calibration data and create a 0 to 1 based velocity.
Each tick
Do a sphere trace to find actors / components of a Wind object type within a couple of meters. Raise an event on the actor so they can process the wind.
Showing breath in cold air or bubbles when underwater
Spawn new sprite emitters for each part of the breath and send them off on a forward vector based on the velocity at that time.
Dust particles
Dust particles are spawned as actors with physics enabled so I can simply apply a directional force.
Blowing dust off an object
Use a mask to hide the dust where it has been blown away. I used a SceneCaptureComponent2d and some hidden geometry painted with decals to create the mask. This could be done much better and smoother using C++, updating the texture for the mask directly and using the mask to calculate the size of the dust cloud when blown.
Candle Flame
A static mesh using a dynamic material instance with world position offset adjusted based on breath velocity.
Cloth
Use a skeletal mesh and NVIDIA APEX SDK to create the cloth. I use a WindDirectionalSource to manipulate the cloth based on the rotation of the hmd and breath velocity. I couldn’t find a way to make the wind source only affect a single actor. Ideally there would be “wind importance volumes” attached to each wind source so we could define the reach and fall off.
Bottle
Just uses a collision box and plays a sound when found by a sphere trace.
Low oxygen area
Note that you can’t detect an inhale. You can play a sound on an exhale, then assume an inhale when nothing is detected. Play around with affecting the volume based on the velocity, or the decay based on the duration of the exhale.
edited:to correct formatting
