[PLUGIN] Speech Recognition

Community Showcase Game Development
23

Hi khoweilip,
I will take a look into an Android implementation.
I may have to branch my code, and change a few things to get it to work with pocketsphinx-android.

In windows based news, I have a further update.

Instructions for Sample Project:

  • Extract the archive to a folder in your Unreal Projects directory (SpeechTest)
  • Open SpeechTest.uproject, and run the project.
  • Hit ‘I’ to initialize, and say any of the following phrases

‘start game’, ‘end game’, ‘open menu’

  • Examine SpeechRecognitionCharacter blueprint, to see the structure.

Phrases are stored as structs. (RecognitionKeyWord)
RecognitionKeyWord’s have a keyword, and a tolerance setting.

Note: Keywords must be built from words that exist in the dictionary. (content/en-us/en-us.dict)

At the moment, I have set Tolerance as an Enumeration, ranging from a very low cut-off, to a very high cut-off.
As a test, try saying the following sentences, one at a time.

  • Being angry about something that happened in the past; a grudge.
  • When something is extremely common and simple to acquire.
  • Someone acting foolish can easily lose his or her money through carelessness or trickery.
  • A task that is simple to complete; similar to the common phrase “as easy as pie.”
  • Something that is extremely expensive; an idiom meaning the price paid was costly, excessively so.
  • When something is incomprehensible due to complexity.

Now, change each of the structs to a very low cut-off, and try again.
The sentences should record false positive for the added phrases.
For simpler phrases, set a high cut-off, and vice versa for a phonetically complex phrase.

At the moment, I sort of tested and guessed with the setting for the tolerances
Here’s what they match to

Instructions for Implementation in your project:

  • From the archive, copy the following to your project

content/model
Plugin

  • In your blueprints, copy the blueprint structure of SpeechRecognitionCharacter.

Would love feedback and suggestions cheers

Known Issues
At the moment, there is a minor issue, where a phrase is spoken eg “start game”, and the phrase is not detected.
The phrase is then spoken again, and then both phrases are detected.
Eg. “start game” “start game”

24

Hi , Sorry I was away from my PC for the last 2 days. I have tested your new update. It worked great. Just that the “start game” did not work quite well for me. When I say “start game”, it came out “end game”.

25

Hi Khoweilip,
At the moment ,this is still very much a work in progress.
No speech recognition will be perfect, but I would very much like to get this as accurate as I can.
Have you tried a higher tolerance setting? I think the default on each of the phrases was high, instead of very high.
If you could send me the audio recordings of the you speaking the phrases, it would certainly help me in testing, and improving the accuracy.
Cheers,

26

I have updated the code-base to support UE 4.10.

27

Hey , I would also be very interested in using this for android projects!

I’m working on a Gear VR game, and having AI respond to speech would be fantastic, given the form factor.
Based on recognized speech/matched words, the AI would select an audio file to respond with. Or, at the least, respond with printed text.
If you can get either scenario working in Android, I would happily pay you for your time.

PM me if interested. Thanks!

28

Is this plugin still alive and being supported?

Does it work on Android ?

Thanks.

29

Hi, Thanks for your great plugin,

I made a C++ project using TopDown Template, and some i using your default command, but when i exit the game in editor, engine stop working and crash…
but i use your sample project and everything is ok on same engine version 4.12.2 …
What’s wrong in my project ?

60a8abc41a087191276043b5be7617d8f3fc3e7f.jpeg

UPDATE :
I forget to shutdown on EndPlay

31

Thanks for your plugin.
I’ve created a blueprint according to your steps to see if it’s work for Chinese.However,after speaking commands,these commands appear very quickly.And after a while,it’s cost of memory increase madly(99% of 32GB).
I’ve set the tolerance of phrases many times.But the result remains the same.Is anyone meet the same problem?

32

Hi ,

Thanks so much for this wonderful plugin! I’ve been using it on desktop and am about to take on an android project that requires speech recognition. Any chance you’ve had some time to work on android support? Thanks again!

33

First, this is awesome work! Second, will it work with 4.15 (on both Mac and Windows?)

Thanks in advance!

34

There’s an available version for Unreal 4.26?
Thanks.

35

Hello, I just want to let you know that I’ve successfully compiled this plugin in UE5. I also changed a bit its architecture, now the SpeechRecognition manager is a Subsystem, not an Actor.

I put the info on how to use it in the Unreal Wiki: Sphinx: Speech Recognition Plugin | Unreal Engine Community Wiki

Thanks!

36

Does it work on andriod ? I kind of revived the andriod build. can you help me debug. the java code is not getting executed. GitHub - ssshammi/sphinx-ue4-andriod: Forked from shanecolb / sphinx-ue4. ssshammi@gmail.com

37

Unfortunately, the original plugin is designed for Windows and Mac only.