Sphinx-UE4: Speech Recognition Plugin [Release v1.0]

It seems my code did not correctly set Sphinx params.
I have committed an update to GitHub.

I will update the sample project in the coming days.

For sphinx params, I have had pretty good results with the following:

**
-vad_postspeech 10
-vad_prespeech 10
-agc “max”
**

I`m sure I can improve my accuracy, with additional tweaking, it’s just 3AM atm :confused:

My game has got nearly 15,000 text dialog segments to it. so it would need to be read from a
database of text ID strings and flags and speaker name. it’s so easy to set it up in windows script language, but hard to do in Unreal. Blueprints are limited. You need a text string database.

Without some context, that made little sense to me.
The methods and classes can easily be called through c++, if you wished to do that instead.

Sort of like this

Speaker ID: Storm0391…“String Text Dialog”…%flag name%…%Flag Counter%…%Audio file pathname%

The Problem is, I don’t understand C++
so i have a huge disconnect between the language that Unreal Uses and the language that I know which is simple script.

In other words you need a unreal coder to decipher my script language into unreal code.

I still don’t understand what you are trying to achieve.
From the fact you mentioned 15,000 text dialog in an earlier comment, it sounds as if you are looking to convert text to audio.
That is not the purpose of this plugin.

I have made a few small improvements/fixes to the plugin, and have created a tag for Version 1.0

The Wiki has been updated, and includes a demo project (as seen in the video below)

Just wondering if there are any news about Android support.

Unfortunately, my spare time has been spent on developing a HTC Vive input mapping application, controlled by javascript scripts (eg. Vive input controlling mouse, keyboard, game-pad).
The port is still on my to do list, but I first want to release a super early alpha of the Vive mapper, for people to try out, before I switch back to working on the Android port of the plugin.

First of all your plugin looks awesome and I would love to see how much it will grow.
I opened the project, then the game map, run the game and then when I tried to give commands from my mic, nothing happened.
I’ve been trying to find solutions all day.
Do I have to compile anything or do anything else before trying to run the project in UE4?
I need any help I can get, because I am really interested in seeing how this plugin works.

Thank you very much!

Hi Tee,
Sorry for the late update, I haven’t checked this forum in a fair while.
The recognition plugin uses the default microphone of the system. Is this the microphone that is being used?
The plugin binaries are included in the demo project, so this shouldn’t be the issue.
Was the project updated to a later version? because this could perhaps cause an issue.
I`ll PM you a link to a pocketsphinx test application in the next day. This can be used to test recognition that mirrors the Game map of the demo project.
This will help isolate if the issue is with pocketsphinx in general, or an issue with the unreal implementation or project.

In other news, I have ported the plugin to android, and am looking for people to try and get the demo project working on their Android devices, and then hopefully implement it within projects of their own. Please try it out.

https://forums.unrealengine.com/comm…ndroid-release

Regards,
Shane

Hi Shane,

Thank you so much for this! It works great.
After playing around with it a fair bit, I decided to implement it into my own project. However I’m getting issues with memory leakage from the node ‘Enable Keyword mode’. Have you experienced this before/have any advice to fix it? :slight_smile:

Thanks!

Edit: just found out the problem which was my silly mistake. Forgot to implement ‘Event End Play’ event. Oops!

Glad to see it was an easy fix :slight_smile: Blah, i really need to add the Shutdown step into the Wiki steps, instead of just the demo project…too many things, too little time >_<

I have some problems with repeating events. When it detects a phrase, sometimes the “Switch on string” it outputs other things besides the phrase spoken. Can you help me? I would like to know also if I can use this plugin in a commercial game. Thankyou.

I managed to get it to work in 4.22 by following the advice in this link. Delete all the generated files (binaries, intermediate, etc.) including the visual studio file in both the main project directory and within plugins > speech recognition, and then attempt to convert the example project to 4.22 by choosing “switch version” in the file explorer or in your Unreal 4.22 Launcher.

Thank you so much for porting this to Android. I cannot wait to get home and try this out on my Quest to see if it works. If it does, you are a true lifesaver!

So after attempting to try the Android ported version on the Oculus Quest, I am sad to say it does not work. I was unsuccessful in bumping the plugin to 4.23. (will probably continue to try that over the next few days) Trying a 4.19 build in the Quest does print a Success for init on event begin play, but does not actually respond to any voice input. I’m not sure if somehow the mic is just not being recognized, if there is a setting somewhere blocking input by default, or if there is some miscommunication within the plugin-to-headset. My mic -does- work in apps like vr chat/etc, so I know I do not have a faulty microphone in my quest. It could be that the 4.19 version does not recognize my quest because (i think) only the newest versions of Ue4 actually do recognize it. (4.22 and 4.23?). If anyone has had any success in getting this to work in the Oculus Quest please message me or reply. I’m desperate to get keyword recognition working for my app!

Hello Shane, i am wondering what is the status on the speech recognition plugin… I got it to work on PC, but on mac… it’s not working at all… I’ve went all the way down to 4.12 but still nothing worked… How can we upgrade the plugin to work on 4.23 or 4.24? thx!!

Marc.

Has anyone got this to work in 4.25?

Hello, the Sphinx for Ue4 from ShaneColb is still UE4 4.12 BUT This one seems to be the same thing, compiled for 4.25! NOTE: IF THIS LINK HELPS YOU, PLEASE COMPLILE A 4.25 PROJECT WITH A .UPROJECT FILE AND ATTACH/EMAIL TO: SchoolHomeVR@gmail.com (I can’t not build solutions for the life of me! But I rarely need to, HOWEVER, in this instance, THERE IS NO .UPROJECT FILE !

for 4.25: https://github.com/MothDoctor/sphinx-ue4

Hello, the Sphinx for Ue4 from ShaneColb is still UE4 4.12 BUT ***This one seems to be the same thing, compiled for 4.25! ***

NOTE: IF THIS LINK HELPS YOU, PLEASE COMPILE FOR ME, A 4.25 PROJECT WITH A .UPROJECT FILE AND ATTACH/EMAIL TO: SchoolHomeVR@gmail.com

(I can’t not build solutions for the life of me! I rarely need to,
HOWEVER, in this instance, THERE IS NO .UPROJECT FILE ! i AM CERTAIN THERE USED TO BE BECAUSE WHEN i LAST TRIED THIS i WAS VERY NE TO ue4 AND WOULD NOT HAVE KNOWN HOW TO ‘BUILD A SOLUTIN’ ANYMOR THAN i DO TODAY (i AM A CHARACTER ANIMATOR FOR THE MOST PART)

i sure would appreciate the help!

https://github.com/MothDoctor/sphinx-ue4