Because it plays fine when you do it before the audio component begins playing, it sounds like maybe an issue with the audio thread messaging and not a fundamental issue with the audio engine.
There could be some edge case with our audio thread you’re hitting that we’ve not run into.
Just so I’m clear on exactly what you’re doing, maybe you can post a video?