TimeSynth not working as expected?

Development Audio
1

Hello,

I’ve watched Dan’s livestream recently and wanted to give TimeSynth a try as the “audio gaps” are a regular annoyance in a title we are working on.

After looking at the provided sample project I attempted to recreate the project in C++. Problem was no music was playing, even though the clip and the synth both showed as valid objects and the Play() function was called. There was also a weird glitch, which was when we stopped play in-editor, the entire engine would freeze indefinitely (it never actually crashed).

I thought maybe the issues stemmed from a complexity in C++. So I built a project (following the sample project) in blueprints and had the exact same results (no audio, infinite freeze).

To better explain what I’m doing, I’m creating both the TimeSynth object and the TimeSynthClip object in the GameInstance, and calling the code from there.

I understand this is a beta plugin, but the sample project works just fine. So what I’m trying to figure out is this: are there other restrictions that should be noted when setting up TimeSynth? Are there some objects that need a strong reference? Is there more documentation on TimeSynth elsewhere?

Thanks,

Blake

2

Can you show your BP setup? Im prototyping a music system and TimeSynth is working for me

3

Sure. So here is my setup in BP. Very straightforward. In the debugger both the Time Synth and Clip show as initialized and properly setup when Play Clip is called (ie the SoundWave is properly pointed to from the LoadSynchronous call).

Here is the C++ code:



UMusicManager::UMusicManager()
{
     TimeSynth = CreateDefaultSubobject<UTimeSynthComponent>(TEXT("MusicTimeSynthComponent"));
}

void UMusicManager::BeginPlay()
{
     FTimeSynthClipSound Sound;
     Sound.SoundWave = Music.BodyPtr.LoadSynchronous();

     TimeClip = NewObject<UTimeSynthClip>();
     TimeClip->Sounds.Add(Sound);
     TimeClip->ClipDuration.NumBars = 80; // TEST
}
4

This looks like it should work. There are 2 things I suggest - make sure you specify a volume group for the clip (just create one) and set the BPM for the TimeSynth with node SetBPM. See if that helps :slight_smile:

5

Hmmmmm, I don’t think this has to do with the TimeSynth persay, I suspect there’s some weird ownership issue on the TimeSynthComponent. The Synth Component is derived from a Scene Component and the expectation is that it would be owned by a Level, so there might be some hang you ran into where the Level is trying to clean up all the objects it has reference to, including the TimeSynth, but can’t because the Game Instance is holding on to it or something like that. That’s my guess.

In my example project, the TimeSynth is owned by an Actor that lives in the Level and is owned wholly by the Level itself. I’m not sure if we’ve done any internal testing with a Synth Component being owned by the Game Instance or being passed to it between level flushes.

Interesting problem. We’ll take a look at that for the future, but in the meantime you should be able to get results just using the TimeSynth Component like any other Scene Component–within the scope of the Level.

EDIT: I forgot we also had some TimeSynth fixes go in for 4.23, so you if you’re on 4.22, then I would try 4.23 as well.

6

I did confirm that the TimeSynth has to be in the level to work properly. Not exactly sure why it needs the level, other than it’s derived from Scene Comoponent.

Maybe in the future it will be level independent. Was hoping it could replace the current music manager solution we have that uses Audio Components because of their ability to persist between levels.

Thanks for your help Dan :slight_smile:

7

I tried to work with it several times, but with no luck. I hope, the problem can be found. As it is not too simple to work without this.
https://i.ytimg.com/vi/ou9n05Ufns8/maxresdefault.jpg
https://www.traveldailymedia.com/best-holiday-destinations-for-students-in-summer/

8

Seems like the only thing the UTimeSynthComponent needs from a USceneComponent is the transform for calculating distance in PlayClip(). And to be totally honest it doesn’t look like USynthComponent needs it that much either. I’d gladly trade distance for audio that persists between level transitions. Surely abstracting USynthComponent away from USceneComponent to UActorComponent (or even UObject) is possible here…

9

The SynthComponent wraps an Audio Component which is a Scene Component derived object. The Audio Component is where you get all of your SoundBase behavior like Submix Routing, Sound Class, Source Effect sends, etc. Though they have to be manually forwarded by the implementation.

10

Hey Dan, Thanks for the reply. And ok gotcha, makes sense. Didn’t realize it was so dependent on the Scene Component.

Here is what is confusing to me: the Audio Component derives from a Scene Component but is able to persist (or not be flushed) between level transitions, thus preserving audio playback. But a Synth Component (which also derives from a Scene Component) is not able to do this. I’m guessing this “persistent” functionality could be added to the Synth Component since an Audio Component can do it and they share the same parent (and thus the Time Synth Component would inherit it as well).

The Time Synth is great for our use of soundtracks as we want to jump in or out of parts of the track based on quantization and dynamic player interactions in the environment. For example, a player enters a dangerous area, we want to switch to a more ominous soundtrack. Doing this transition on beat makes the audio sound uniform and correct, not jarring. The Time Synth does this with incredible accuracy, precision, and immediacy that a normal Audio Component doesn’t have.

But with soundtracks being longer and needing to persist or have time to “finish”, it is quite jarring to have them end when a loading screen is displayed (aka a level transition is occurring). An Audio Component can can avoid this issue somehow, although I’m not quite sure how it achieves this while being a Scene Component other than I know you need to pass the Audio Component a new World object.

Maybe I am overcomplicating the solution to this problem. When I get back to my computer I will try to see if it’s possible to do something like “TimeSynthComponent->AudioComponet->bIgnoreForFlusing” and just have it owned by the Game Instance. But I’m not sure how the “Scene” part of Synth Component would work when traveling.

Anyways, thanks for the comments. If you have any advice for us on the above please let me know :slight_smile: