So my problem was not with SetGamePause, but with SetPause function of AudioComponent. I did a fix in AudioDevice where it updates the PlaybackTime of the ActiveSound, to only update if the ActiveSound is not Paused. That fixed my issue. Thanks anyway