Quartz delaying playback of metasounds

Development Audio
, ,
1

Hello

I have a few questions regarding audio and Quartz.

I’m building a music system using Quartz, and I’m using it to queue up multiple (at most around 8) music layers to play at a specific point. A music layer consists of a main wave asset and optionally an entry wave asset (at most a few seconds long, and used for crossfading and in general smoothing the transition).

Early on when building this out, I noticed that sometimes Quartz has trouble starting the metasounds with entry wave assets in time. And I read on UDN (I think) that it’s because Quartz needs to go through the wave assets used in the metasound graph. So I gave it some buffer time between the queue time and the quantized play timing, and that seemed to work.

But now, it’s starting to happen again, but much more randomly. Sometimes it’s playing all my layers at the correct time, but other times it’s only playing metasounds with main wave assets at the correct time, while metasounds with both main and entry wave assets are delayed by up to 10 seconds (probably until the next time the musical boundary occurs again).

For now I’ve increased the buffer time from 100 ms to 200 ms, and that seems to work, but this is starting to worry me - especially the random nature of it.

Does the platform affect the performance of Quartz, and would I need to increase the buffer time even more for e.g. Switch 2? If I increase the buffer time too much, the system’s flexibility start to suffer.

Is there any way to configure Quartz to alleviate this problem? I haven’t been able to find anything in project settings.

Would it help if I staggered the queueing of Quartz events slightly? At the moment I’m queueing up 8 metasounds, a handful of notifies, 1 transport reset and potentially a BPM change or a StartOtherClock - at the same time - worst case.

Am I perhaps reaching the limits for what Quartz was designed to do, and do I need to do some things differently/scale back?

Thanks!

[Attachment Removed]

2

There are some diagnostic things one can do to isolate the source of the delay. But first to clarify a few things:

>> “(I think) that it’s because Quartz needs to go through the wave assets used in the metasound graph”

No, Quartz basically has the MetaSound get ready to play as normal, but pauses it until the buffer it should start rendering, then runs the output of the MetaSound through a delay line for sub-buffer accuracy. Quartz doesn’t know anything about the waves inside of a MetaSound graph.

>> “For now I’ve increased the buffer time from 100 ms to 200 ms”

What does this mean exactly? Is this the metasound buffer size? The audio mixer buffer size?

>> “but other times it’s only playing metasounds with main wave assets at the correct time, while metasounds with both main and entry wave assets are delayed by up to 10 seconds”

How consistent is this? could you include a screenshot of a metasound graph with both main and entry wave assets? How are you scheduling the time difference between the main & entry sounds withing a single metasound source? (Or are they supposed to play at the same time?)

How far ahead in musical timing are you scheduling your sounds? If PlayQuantized for multiple sounds is getting called right before those sounds are supposed to start, it is possible for some of the sounds to miss the cutoff and start on the next occurrence of that musical deadline. A large buffer size can actually make this problem worse in some cases.

You can also mark the soundwaves’ loading behavior to Force Inline. This uses more memory, but would eliminate latency due to encoded audio cache misses. (though this would cause a slight delay, not multiple seconds.)

Most likely you need to call play quantized earlier.

[Attachment Removed]

3

Thanks for your quick reply!

>> “No, Quartz basically has the MetaSound get ready to play as normal, but pauses it until the buffer it should start rendering, then runs the output of the MetaSound through a delay line for sub-buffer accuracy. Quartz doesn’t know anything about the waves inside of a MetaSound graph.”

Ah, thanks for the clarification. That makes a lot of sense. But is it then the metasound preparing itself for playback that causes it to miss its start time as scheduled by Quartz? I’m able to reproduce a delay every time I test with a metasound with both a main and an entry wave asset, when the clock has not yet been started and the settings for quantized playback is set to fire on clock start.

>> “could you include a screenshot of a metasound graph with both main and entry wave assets? How are you scheduling the time difference between the main & entry sounds withing a single metasound source? (Or are they supposed to play at the same time?)”

[Image Removed] [Image Removed]This is the layer metasound graph. I’m using the OnNearlyFinished trigger of WavePlayer to concatenate the entry and the main wave asset, if there is an entry wave asset in the input WaveAssets array.

>> “What does this mean exactly? Is this the metasound buffer size? The audio mixer buffer size?”

>> “How far ahead in musical timing are you scheduling your sounds?”

What I call “buffer time” is exactly this - how far ahead of the musical timing that I call PlayQuantized. Sorry about the confusing naming. So at the moment, I’m giving Quartz 200 ms to crunch the scheduling of the metasounds. I started at 100 ms when I discovered the delay that I described above. But lately it has started to happen again more randomly.

>> “A large buffer size can actually make this problem worse in some cases.”

Our Callback Buffer Size in the Windows platform settings is 1024.

>> “You can also mark the soundwaves’ loading behavior to Force Inline. This uses more memory, but would eliminate latency due to encoded audio cache misses. (though this would cause a slight delay, not multiple seconds.)”

The sound waves are set to Prime on Load. I’ll test it with Force Inline.

But you’re probably right that is it caused by the sounds being scheduled too late, since the delay seems to match the next occurrence of the musical boundary. I recently added queue cancellation functionality, which meant that I had to push the PlayQuantized calls much closer to the musical boundaries (but still spaced by at least 200 ms). Perhaps the timer is overshooting and PlayQuantized gets called too late.

[Attachment Removed]

4

Yes, if the delay matches the next occurrence of the musical boundary the call is getting evaluated on the audio render thread after your desired target time. More lead time is the answer.

I don’t know the nuances of your game, so this may be a naïve suggestion, but could your cancellation mechanism exist in the metasound instead of delaying your call to “play quantized?”

You could even add a slight delay in the metasound graph(s). If they are all delayed the same all the synchronization would be intact (just universally latent), and then you’d have some more wiggle room for Stopping the sound in case where you want to cancel.

[Attachment Removed]

5

Yeah, increasing the lead time to 200 ms actually turned out to eliminate the random delays. What confused me was another unrelated bug in my queue cancellation code that resulted in music layer UObjects being cleaned up by GC (because they were referenced in a lambda used by a timer and not cached in my subsystem).

The current 200 ms lead time is acceptable I think, in terms of how fast the system can react to a music state change. I only got a little worried if I would have to increase that time even more for slower platforms (like Switch 2), and by how much.

>> "I don’t know the nuances of your game, so this may be a naïve suggestion, but could your cancellation mechanism exist in the metasound instead of delaying your call to “play quantized?”

You could even add a slight delay in the metasound graph(s). If they are all delayed the same all the synchronization would be intact (just universally latent), and then you’d have some more wiggle room for Stopping the sound in case where you want to cancel."

It’s a good suggestion, but it wouldn’t work in my case unfortunately. I actually managed to cancel an already Quartz-scheduled audio component by resetting its Sound property, as this triggers the built-in cancellation of the PlayQuantized command. But I would still need to cancel the other commands (notifies, transport reset, BPM change and StartOtherClock), and cancellation of these seems to be unimplemented in the Quartz code.

[Attachment Removed]

6

Good to hear! Feel free to re-open if you hit a limit between lead time and gameplay :slight_smile:

[Attachment Removed]