[FREE] AzSpeech plugin: Text-to-Speech, Speech-to-Text and more with Microsoft Azure Cognitive Services

Okay that’s actually incredible, thanks for clearing that up - i was convinced we were supposed to be using TextToSoundWave so we can get the current time of the sound component and sync with those viseme data. That’s a lot clearer now, thanks. Will dedicate some time to do a deep dive into this plugin today and give the docs a good read for this feature now that I know what to use. What a time to be alive

2 Likes

Hello there :slight_smile:

I’m sometimes getting this error when I stop playing in editor. Is there any way to prevent it from crashing?

UnrealEditor_AzSpeech!FAzSpeechRunnableBase::Exit() [D:\Solidlake\Plugins\AzSpeech\Source\AzSpeech\Private\AzSpeech\Runnables\Bases\AzSpeechRunnableBase.cpp:72]
UnrealEditor_AzSpeech!FAzSpeechSynthesisRunnable::Exit() [D:\Solidlake\Plugins\AzSpeech\Source\AzSpeech\Private\AzSpeech\Runnables\AzSpeechSynthesisRunnable.cpp:76]
UnrealEditor_Core!FRunnableThreadWin::Run() [D:\build++UE5\Sync\Engine\Source\Runtime\Core\Private\Windows\WindowsRunnableThread.cpp:152]

nvm, I think I fixed it.

By just manually ending the task/s on endplay

1 Like

It worked on 1.4.6, thank you very much

1 Like

sent to relevant logs via mail

1 Like

Wow! Thank you for the information, I will try to replicate the issue and find a solution. The tasks already wait for the End PIE event to force stop. But it seems like there was some issue happening.

If it happens again, could you please send me the application logs?

If needed, you can send it to my email or as a private message. Email: contatolukevboas@gmail.com

1 Like

I’ll check as soon as possible! Thanks! : )

1 Like

Hi. When I used the latest AzSpeech1.46 on UE5.1, I found some serious bugs, all the blueprint nodes involved in wave conversion would have assertion errors, causing the engine to crash. For example, the node in the figure below.



The plug-in settings are as follows:

After I tested, I found that blueprint nodes related to sound wave conversion such as “Text to Sound Wave”, “SSML to Sound Wave”, “Convert Audio Data to Sound Wave” and so on will have the same assertion error.
The picture below is the computer configuration I use, and the GPU is RTX3070.

I hope this problem can be solved as soon as possible, thank you.

1 Like

I’m checking right now, I also replied to the private message! : )

thank you very much.

1 Like

If the engine crashes the moment it’s about to play the sound, try not to put it under a directory with Chinese Characters should work

2 Likes

AzSpeech v1.5.0

Changes

  • Fix error logging
  • Add new setting to enable usage of private endpoints
  • Set audio compression to not occur in transient sound waves
  • Fix audio compression occurring more than one time
  • Fix audio compression being called outside audio thread
  • Separate tasks signals to avoid the completion delegate sending wrong data (this fixes the ‘sound wave with 0 duration…’ issue)
  • Set to only broadcasts be called in game thread
  • Set the runnable threads to try lock on exit to avoid crashes after End PIE
  • Add comments in some functions
  • Move Viseme Data to a separate header
  • Create new structure: Animation Data
  • Added new functions to extract Animation JSON Data from Viseme Data

Screenshots

image

Full Changelog: Comparing v1.4.6...v1.5.0 · lucoiso/UEAzSpeech · GitHub

I am looking forward to the new version very much, but I see that the marketplace is still not updated. How long is the review time for EPIC under normal circumstances?

1 Like

They just approved the update! : )

Upcoming changes

  • All tasks will allow custom options
  • The options structure are initialized with the default settings from Project Settings
  • New function to modify the default settings
  • Existing tasks marked as deprecated will not be removed immediately, don’t worry xD

Currently not tested, only available in this working branch: GitHub - lucoiso/UEAzSpeech at feature/CHANGE-SETTINGS-RUNT-155
Issue: Add option to change the values of Azure Settings at runtime · Issue #155 · lucoiso/UEAzSpeech · GitHub

2 Likes

Great, I have used the new version, the previous problem has been solved, thank you very much. However, when I used the latest blueprint node Extract Animation Data From Viseme Data Array, I found that the length returned by the BlendShape array in different frames was inconsistent. Some frames were only 12, and some frames were 59, but according to the Microsoft Azure standard, it should be a constant 55? I don’t understand here, please tell me why.

1 Like

我也在学习使用这个插件提取数据,现在头大,有问题向您请教,能加下您的微信么?

I encountered some problems when using this plug-in to obtain animations. I hope to ask you for advice. Can you add your WeChat?
我在使用这个插件获取动画时,遇到了一些问题,希望能向您请教,能加下您的微信么?

Sorry, 我也还在学习中,你在这里问插件作者或给他发邮件他会很热心帮你解答。

AzSpeech v1.6.0

Changes

  • Settings related to tasks behaviors and characteristics are now mutable: You can change it at runtime
  • All Azure related tasks now accept Custom Options: All tasks have 2 versions (with Default Options & with Custom Options)
  • Some FStrings properties were transformed into FNames
  • Updated Azure Speech SDK to v1.26.0
  • Add new setting: Enable Debugging Prints
  • Move debugging prints to recognition& synthesis task base
  • Add latency properties to tasks: Exposed with getters + Shown at debugging log & print
  • Add new functions
  • Add missing _API macro to structures

Screenshots

Debugging Logs: Recognition

image

Debugging Logs: Synthesis

image

2 Versions of each Task

Mutable Settings: Default Options

image

Separated by Categories

image

More Functions: Get Latency Informations

More Functions: Get Extracted Animation directly from the Task

Settings Page

Sample Project Updated: SpeechGPT

.