I’m considering moving the rhythm game prototype I’m developing into UE and am concerned with the same questions being discussed here so I thought I’d jump in with two cents. I haven’t put much time into researching UE’s audio as frankly the documentation is rather opaque so forgive me if I’m getting the wrong end of the stick here.
There’s a tradition of rhythm game sync design (for the typical rhythm game that syncs user input with the playback of an audio file) which stresses against using any kind of system or game-wide clock as a sync source. Instead the “ground truth” of sync must come from the audio subsystem’s record of how long it has been playing the audio file (minus its reported audio latency). If the audio engine only updates this time in chunks, the time since the last update can be added to the reported time (this is how Godot does it).
This sync is constantly monitored by the game and used to generate a series of beat events which any user input can be measured against (usually you’d keep the song_position in seconds as a float and the song_position_in_beats as a floored integer, derived by dividing by secs_per_beat).
So I’m wondering - does Quartz replace the need for generating a beat event stream by automatically integrating with the audio subsystem and maintaining a beat (while taking both subsystem time-chunking and latency into account)? Failing that, does UE5 offer a way of accessing the necessary audio playback timing info to do it in the usual way?
Looking at the one article in the help, it does sound like Quartz Metronome will provide an accurate record of audio events, but the article is focused on starting playback on a sample accurate basis rather than providing a hyper-accurate measure of how far playback has progressed.
Once we have a “ground truth” of beat timing established, any user input can be measured against this and any synced animation can be lerped between states. All updates of animation positions etc need to be lerped or incremented with care taken to avoid letting frame timing have any creeping influence - see blogs below for more detailed and far superior explanations of this.
Below are some blog posts covering the logic of sync in rhythm games - how to create and maintain it as well as the pitfalls to avoid.
https://shinerightstudio.com/posts/music-syncing-in-rhythm-games/
This one is written by the maker of the excellent Rhythm Doctor:
https://www.reddit.com/r/gamedev/comments/2fxvk4/heres_a_quick_and_dirty_guide_i_just_wrote_how_to/