# Speech Recognition Plugin - Sphinx-UE4 **URL:** \ **Category:** Plugins\ **Tags:** unreal-engine\ **Created:** [December 18, 2015, 3:37pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375 "2015-12-18T15:37:30Z")\ **Posts on this page:** 20\ **Page:** 3
**Author:** ![ShaneC](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@ShaneC](https://forums.unrealengine.com/u/ShaneC)\ **Post date:** [April 15, 2016, 9:41pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/41 "2016-04-15T21:41:48Z")
motorsep, yes, I am definitely looking into the possibility of using the Android native speech recognition. I will keep everyone posted. If anyone takes the existing plugin, and makes any noteworthy changes to the plugin, please don’t hesitate to contact me 🙂 ---
**Author:** ![motorsep](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/motorsep/32/918698_2.png) [@motorsep](https://forums.unrealengine.com/u/motorsep)\ **Post date:** [May 4, 2016, 12:25pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/42 "2016-05-04T12:25:22Z")
Just wondering how it’s going with Android version :o ---
**Author:** ![ShaneC](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@ShaneC](https://forums.unrealengine.com/u/ShaneC)\ **Post date:** [May 8, 2016, 12:48am UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/43 "2016-05-08T00:48:53Z")
Poorly, not because of having tried, and failed…simply because my Vive came, and all my spare time programming/UE work has been spent on that. I`ll sit down and work on it solidly shortly 😕 Sorry. ---
**Author:** ![motorsep](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/motorsep/32/918698_2.png) [@motorsep](https://forums.unrealengine.com/u/motorsep)\ **Post date:** [May 8, 2016, 12:59am UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/44 "2016-05-08T00:59:29Z")
Oh, ok. Understood and thanks. ---
**Author:** ![MatzeOGH](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@MatzeOGH](https://forums.unrealengine.com/u/MatzeOGH)\ **Post date:** [May 8, 2016, 8:23pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/45 "2016-05-08T20:23:46Z")
Would it be possible to analyze sound files and generate phonemes with this plugin? I’d like to drive a lip-sync rig with those phonemes. If it is possible can you point me in the right direction to get started with solving this problem. ---
**Author:** ![ShaneC](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@ShaneC](https://forums.unrealengine.com/u/ShaneC)\ **Post date:** [May 9, 2016, 1:29am UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/46 "2016-05-09T01:29:35Z")
If you have a read earlier in this thread, both myself and n00854180t have experimented with phoneme detection. However, the results have been wildly unreliable, even with their test application (outside of UE). If you’d like, I can send you the sample code, to have you see for yourself/experiment? Are you capable of recompiling the plugin? It’s pretty simple. All you really need to do, is right click the .uproject file (of a project with the plugin), then regenerate project files. Then open the project in visual studio. The plugin source should now be visible, and you should just be able to build the project. ---
**Author:** ![MatzeOGH](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@MatzeOGH](https://forums.unrealengine.com/u/MatzeOGH)\ **Post date:** [May 17, 2016, 5:39pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/47 "2016-05-17T17:39:30Z")
How do I get the defected phonemes form shinx? A output like the following would be awesome: G 0.3 0.5 OW 0.5 0.7 F 0.6 0.9 ---
**Author:** ![JTensai](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@JTensai](https://forums.unrealengine.com/u/JTensai)\ **Post date:** [May 20, 2016, 10:46pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/48 "2016-05-20T22:46:22Z")
First of all, thank you so much for this Plugin it is great to have something like this available. I have been able to get the plugin working in unreal 4.11.2 (Though I am having a lot of difficulty with accuracy of phrases) I am however having a lot of issues when it comes to packaging the game. Here are the things I have found so far: 1. Cannot get the plugin to work with a blueprint only project. This seems to be something that Epic is aware of, but haven’t done anything about. Only solution: Add a C++ class to the project so it builds it as such and includes all plugins. 2. Must package for Windows 64 bit, trying 32 bit will fail the build process mentioning that it cannot access SphinxBasex86.lib (it doesn’t exist) 3. Once the game has been packaged successfully it still will not launch from the executable. In order to get it to launch I must go find two .dll files (pocketsphinx.dll & shpinxbase.dll) and put them into “WindowsNoEditor\PROJECTNAME\Binaries\Win64” At this point the game will launch and is completely playable. However, Init is failing ☹ So even after all that I still have no speech recognition in a packaged version of my game. Any ideas? ---
**Author:** ![ShaneC](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@ShaneC](https://forums.unrealengine.com/u/ShaneC)\ **Post date:** [May 22, 2016, 7:34pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/49 "2016-05-22T19:34:35Z")
Hello JTensai, Sorry for the late response. Regarding your issues with the accuracy, can you please send me a PM including the following - Listing of the phrases you are attempting to recognize, along with the tolerance settings for each. - Audio recording of the phrases being spoken. Then I can take a look. > [@](#): > > 1. Cannot get the plugin to work with a blueprint only project. This seems to be something that Epic is aware of, but haven’t done anything about. Only solution: Add a C++ class to the project so it builds it as such and includes all plugins. Yes, unfortunately this is the case until there’s a change to UE4. I have read recent posts by others who have the same issue (Plugin not packaging in Blueprint only project). I have added a note to the Wiki that a empty C++ class must be added to the project, in order for the plugin to build. > [@](#): > > 1. Must package for Windows 64 bit, trying 32 bit will fail the build process mentioning that it cannot access SphinxBasex86.lib (it doesn’t exist) My mistake, I had only included the 64 bit lib’s and dll’s of the Sphinxbase and Pocketsphinx. I have updated Github Code/Sample Projects to reflect the changes. > **[GitHub - shanecolb/sphinx-ue4: A speech recognition plugin for Unreal Engine...](https://github.com/shanecolb/sphinx-ue4)** > > A speech recognition plugin for Unreal Engine 4. This is essentially a port of Pocketsphinx, to be used within an Unreal Engine project. - GitHub - shanecolb/sphinx-ue4: A speech recognition plugin... > [@](#): > > 1. Once the game has been packaged successfully it still will not launch from the executable. > In order to get it to launch I must go find two .dll files (pocketsphinx.dll & shpinxbase.dll) and put them into “WindowsNoEditor\PROJECTNAME\Binaries\Win64” This should be fixed by my latest check-in to the Git repo. The dll’s are now copied during packaging. Note, you have to add “model” as a folder to copy during the packaging process, otherwise the language model is not found (probably what was happening in your case, after you copied across the dll’s). ![Screenshot1.PNG](https://d3kjluh73b9h9o.cloudfront.net/original/4X/1/9/a/19a5bfa1b35f1b2d7992c0a2f5881af831f2a1fd.png) Let me know if you still are encountering issues. ---
**Author:** ![n00854180t](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@n00854180t](https://forums.unrealengine.com/u/n00854180t)\ **Post date:** [May 22, 2016, 7:54pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/50 "2016-05-22T19:54:15Z")
@ - So I’ve started very early work on porting Oculus’ Lip Sync library to a UE4 plugin here [GitHub - ChairGraveyard/ovrlipsync-ue4: Unofficial Plugin for Oculus Lip Sync](https://github.com/ChairGraveyard/ovrlipsync-ue4) It’s only for the phoneme recognition, but it does a much better job than Sphinx, at least out of the box. Right now that repo still has a bunch of stuff from Getnamo’s Hydra plugin, which I was using as a base, and it’s not functional at all. I have set up the enums and the DLL exports though. Usage is all tied up in the C# scripts in the Unity package, but it seems pretty straight forward. ---
**Author:** ![motorsep](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/motorsep/32/918698_2.png) [@motorsep](https://forums.unrealengine.com/u/motorsep)\ **Post date:** [May 23, 2016, 2:13pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/51 "2016-05-23T14:13:19Z")
@n00854180t Nice! I was wondering if anyone ever going to implement OVR LipSync into UE4 🙂 @ So, not a chance for Android version with full BP exposure? Thanks ---
**Author:** ![ShaneC](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@ShaneC](https://forums.unrealengine.com/u/ShaneC)\ **Post date:** [May 23, 2016, 2:17pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/52 "2016-05-23T14:17:07Z")
The Android port is coming, i have just been having some difficulties with Android packaging and how the libraries I am building link (read: winging it). Baring some unforeseen events, I should hopefully have something available after the weekend Unfortunately, I have been flat-out with work and life. As I don’t work as a game dev, and this just slots in when I have some free time. ---
**Author:** ![motorsep](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/motorsep/32/918698_2.png) [@motorsep](https://forums.unrealengine.com/u/motorsep)\ **Post date:** [May 23, 2016, 2:27pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/53 "2016-05-23T14:27:02Z")
> [@;534739](#): > > The Android port is coming, i have just been having some difficulties with Android packaging and how the libraries I am building link (read: winging it). > Baring some unforeseen events, I should hopefully have something available after the weekend > Unfortunately, I have been flat-out with work and life. > As I don’t work as a game dev, and this just slots in when I have some free time. Sweet! Totally understandable. Currently I am in the same boat, so to speak. ---
**Author:** ![JTensai](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@JTensai](https://forums.unrealengine.com/u/JTensai)\ **Post date:** [May 23, 2016, 8:22pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/54 "2016-05-23T20:22:40Z")
Thank you ! That fixed my build problems 😃 ---
**Author:** ![BigVulpes](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@BigVulpes](https://forums.unrealengine.com/u/BigVulpes)\ **Post date:** [June 3, 2016, 3:47pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/55 "2016-06-03T15:47:23Z")
Hi, first of all thanks for the Plugin, it is awesome. I have a problem and hope somebody can help me out. I need to access the exact point in time when the microphone detects any input. I figured it has to be the variable “Utt\_started” in the script “SpeechrecognitionWorker.cpp”, but how can I turn this variable into a node in a blueprint? Tried several thing but none of them worked out. ---
**Author:** ![n00854180t](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@n00854180t](https://forums.unrealengine.com/u/n00854180t)\ **Post date:** [June 3, 2016, 6:44pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/56 "2016-06-03T18:44:04Z")
@BigVulpes - the best way to do it would be to copy the existing WordSpoken event stuff then rename it to your own name (like UtteranceStarted) and set it to fire off when utt\_started is set to true (it’s just a bool if I remember right). * * * Progress on the Oculus Lip Sync lib port. It’s not done yet but all the boilerplate plugin code is set up, now just have to implement passing the audio data and firing the event, and test it out. ---
**Author:** ![motorsep](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/motorsep/32/918698_2.png) [@motorsep](https://forums.unrealengine.com/u/motorsep)\ **Post date:** [June 3, 2016, 7:20pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/57 "2016-06-03T19:20:54Z")
Having voice input coupled with OVRLipSync would be crazy insanely awesome in Gear VR !! ---
**Author:** ![BigVulpes](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@BigVulpes](https://forums.unrealengine.com/u/BigVulpes)\ **Post date:** [June 3, 2016, 8:06pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/58 "2016-06-03T20:06:26Z")
Thanks for the quick reply. I should have said that i am a beginner in c++ ☹ However i did rename the Wordspoken event (see pictures). But if I try to recompile the plugin in ue4 i get error messages (see pictures). Can you point me to where i did something wrong? ![f5eb6cbbaf042e7e194b2afe7da0d69a3c0c1cf1.jpeg](https://d1ap1mz92jnks1.cloudfront.net/images/transparent.png) ![Speechrecoactor.JPG](https://d3kjluh73b9h9o.cloudfront.net/original/3X/c/d/cd6f26a6086fce469f78fc89acd42acaa1da371d.jpeg) ![a0f77add176a626338e6c8dda4dbf44928336d25.jpeg](https://d1ap1mz92jnks1.cloudfront.net/images/transparent.png) ![error 2.JPG](https://d3kjluh73b9h9o.cloudfront.net/original/3X/6/7/67523ef85c645484fc33ed29eb52f41a37315b8d.jpeg) ---
**Author:** ![n00854180t](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@n00854180t](https://forums.unrealengine.com/u/n00854180t)\ **Post date:** [June 3, 2016, 8:47pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/59 "2016-06-03T20:47:52Z")
@BigVulpes - Instead of _just_ renaming them, copy each function first, then rename it, but leave the original WordSpoken stuff intact. The idea is to make an entirely new event to fire off. You’ll need to copy the relevant bits in both the .h (declarations) and .cpp (definitions). If you’re still having problems I’ll give an example. ---
**Author:** ![n00854180t](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@n00854180t](https://forums.unrealengine.com/u/n00854180t)\ **Post date:** [June 3, 2016, 11:24pm UTC](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375/60 "2016-06-03T23:24:57Z")
@ - BTW, is there a specific reason you’re using the Sphinx provided methods (through ALSA) of getting at the microphone data (other than it being convenient/and the way that’s used in the examples)? I was looking into it and was initially going to try and use those as well but figured I’d look into getting access to the microphone samples via Unreal’s API, and found this: [https://quoteunquotestudio.com/2015/02/22/microphone-input-in-ue4/](https://quoteunquotestudio.com/2015/02/22/microphone-input-in-ue4/) I haven’t tested the OVR Lip Sync plugin at all yet but it’s using that method and is just about ready. * * * [https://github.com/ChairGraveyard/ovrlipsync-ue4](https://github.com/ChairGraveyard/ovrlipsync-ue4) The lastest/should work version is on the Github repo now, btw. Again, totally untested! I’ll be testing it this weekend and updating it with an example using the mesh that Oculus provided. [Previous page](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375.md?page=2) [Next page](https://forums.unrealengine.com/t/speech-recognition-plugin-sphinx-ue4/50375.md?page=4)