Custom data importing

Development Programming & Scripting C++
1

I need an audio visualizer, and the problem with UE4 is that the engine remove PCM data from the original wav file. So I’ve this idea import a file.wav and then store the parsed data into new file format and read it during run time.

It’s possible to implement a class that imports this file as an asset of game? thank in advance

2

Yes, this is possible, though it is a fair amount of work. The problem with sound assets is that the engine is hard coded to expect OGG data in many places dealing with audio. I know this because I’m making a FLAC importer, and it contains a few too many workarounds that shouldn’t be necessary for my taste :wink: Though, dealing with raw PCM data will be easier than trying to replace the compressor entirely.

A general approach you could take is the following:

  • Create a plugin with two modules: one runtime module for audio processing, and one editor module for asset importing. See the SoundMod plugin in Engine\Plugins\Runtime\SoundMod for a great example that does exactly this.
  • Your asset import factory must declare it import type as USoundWave::StaticClass() and its file extension as “.wav”. Essentially you can just copy the whole USoundFactory and/or USoundSurroundFactory from EditorFactories.cpp for this (that’s the default .WAV importer). The FactoryCreateBinary() method is what creates a USoundWave object and copies the PCM data to it. One important thing is that you must set the UFactory’s ImportPriority to some value higher than DefaultImportPriority. This ensures that your importer will be used instead of the default one when you import a .WAV file.
  • For the runtime module, I suggest making a UObject class that inherits from USoundWave, so you’'ll have say a ‘UPCMSoundWave’ class. The PCM data will be contained in the RawData member variable. There are two ways that I can think of to feed the samples to the audio device:

  1. Set CompressionName to ‘NAME_None’ and ResourceData to nullptr so that the DecompressionType will be set to DTYPE_Native by FAudioDevice during initialization. This way the audio device will skip decompressing the sound wave as OGG and use the raw PCM data instead.

  2. Set CompressionName to anything other than OGG, set bProcedural to true and override USoundWave::GeneratePCMData(). GeneratePCMData() will be called by the engine to request samples for playback. You can take a look at USoundWaveProcedural to see how it works (but don’t subclass it, because it is transient and doesn’t retain any data). In your case the function would simply copy the samples you already have to the output buffer provided by the device.

Either way will work, so pick whichever you like. The main problem with subclassing USoundWave however isn’t feeding the samples to the audio device, it’s stopping USoundWave from stomping all over your data and/or constantly trying to compress and decompress your stuff as OGG. So you’ll have to override a lot of USoundWave methods to keep your PCM data safe from deletion and (de)compression. Basically you’ll just have to comb through the USoundWave code and override anything you find related to loading, serialization and resource management.

The SANE way to do this would of course be to implement an FAsyncAudioDecompress class that can deal with whatever type of audio format you want, replacing the OGG decompressor. But unfortunately USoundWave’s decompressor instance constantly gets deleted and recreated by the audio device buffer, and there is no way to tell it which decompressor to use. So that’s a dead end. The same goes for making a custom compressor: you can create an IAudioFormat module perfectly fine, but letting the engine know about it is another story.

3

Thank alot for your response, Your approach is perfectly what i want to do.

My Idea is take a file.wav and change the extension to something like file.pcmwav.

When I import this file the engine automatically calls the class that i’ve written. The importer will store only the frequency spectrum of the audio. This asset will used only to animate the visualizer, the Audio will reproduced normally by the engine with a normal file.wav.

So as you said what I need to do is create a editor asset importer and runtime asset parser.

What I need to know is that how to create the importer function, I’ve looked to the SoundMod plugin but I’ve not understand what’s the importer function.

Can you please explain me in short how can I inform the engine that there is a new file format that support? It’s enough the name of a function/class that do this so I can study how it works.

Thanks again

4

There is already an audio visualizer plugin and I believe is free

5

Yes, but works only with editor

however i’ve found this http://www.wraiyth.com/?p=209 that is what I need to create another asset

6

Yep, that explains it pretty well. Another (very well hidden) piece of documentation you may want to check out for more details is in Engine\Documentation\Source\Programming\Assets\CustomImporters. For some reason the docs for this don’t seem to be on the documentation site.

7

No, cedric included functionality in cooked builds: https://forums.unrealengine.com/showthread.php?94974-eXi-s-Sound-Visualization-Plugin-(works-in-cooked-builds)-4-11-UPDATE-EXAMPLE-MAP-RELEASED

8

This plugin is awesome, the only problem is that it need an external directory to store the wave files to load at runtime… and I don’t like this…