motorsep, yes, I am definitely looking into the possibility of using the Android native speech recognition.
I will keep everyone posted.
If anyone takes the existing plugin, and makes any noteworthy changes to the plugin, please don’t hesitate to contact me 
Just wondering how it’s going with Android version :o
Poorly, not because of having tried, and failed…simply because my Vive came, and all my spare time programming/UE work has been spent on that.
I`ll sit down and work on it solidly shortly 
Sorry.
Oh, ok. Understood and thanks.
Would it be possible to analyze sound files and generate phonemes with this plugin? I’d like to drive a lip-sync rig with those phonemes. If it is possible can you point me in the right direction to get started with solving this problem.
If you have a read earlier in this thread, both myself and n00854180t have experimented with phoneme detection.
However, the results have been wildly unreliable, even with their test application (outside of UE).
If you’d like, I can send you the sample code, to have you see for yourself/experiment?
Are you capable of recompiling the plugin? It’s pretty simple.
All you really need to do, is right click the .uproject file (of a project with the plugin), then regenerate project files.
Then open the project in visual studio.
The plugin source should now be visible, and you should just be able to build the project.
How do I get the defected phonemes form shinx?
A output like the following would be awesome:
G 0.3 0.5
OW 0.5 0.7
F 0.6 0.9
First of all, thank you so much for this Plugin it is great to have something like this available.
I have been able to get the plugin working in unreal 4.11.2 (Though I am having a lot of difficulty with accuracy of phrases)
I am however having a lot of issues when it comes to packaging the game. Here are the things I have found so far:
- Cannot get the plugin to work with a blueprint only project. This seems to be something that Epic is aware of, but haven’t done anything about. Only solution: Add a C++ class to the project so it builds it as such and includes all plugins.
- Must package for Windows 64 bit, trying 32 bit will fail the build process mentioning that it cannot access SphinxBasex86.lib (it doesn’t exist)
- Once the game has been packaged successfully it still will not launch from the executable. In order to get it to launch I must go find two .dll files (pocketsphinx.dll & shpinxbase.dll) and put them into “WindowsNoEditor\PROJECTNAME\Binaries\Win64”
At this point the game will launch and is completely playable. However, Init is failing 
So even after all that I still have no speech recognition in a packaged version of my game.
Any ideas?
Hello JTensai,
Sorry for the late response.
Regarding your issues with the accuracy, can you please send me a PM including the following
- Listing of the phrases you are attempting to recognize, along with the tolerance settings for each.
- Audio recording of the phrases being spoken.
Then I can take a look.
Yes, unfortunately this is the case until there’s a change to UE4. I have read recent posts by others who have the same issue (Plugin not packaging in Blueprint only project).
I have added a note to the Wiki that a empty C++ class must be added to the project, in order for the plugin to build.
My mistake, I had only included the 64 bit lib’s and dll’s of the Sphinxbase and Pocketsphinx.
I have updated Github Code/Sample Projects to reflect the changes.
This should be fixed by my latest check-in to the Git repo.
The dll’s are now copied during packaging.
Note, you have to add “model” as a folder to copy during the packaging process, otherwise the language model is not found (probably what was happening in your case, after you copied across the dll’s).
Let me know if you still are encountering issues.
@ - So I’ve started very early work on porting Oculus’ Lip Sync library to a UE4 plugin here GitHub - ChairGraveyard/ovrlipsync-ue4: Unofficial Plugin for Oculus Lip Sync
It’s only for the phoneme recognition, but it does a much better job than Sphinx, at least out of the box. Right now that repo still has a bunch of stuff from Getnamo’s Hydra plugin, which I was using as a base, and it’s not functional at all. I have set up the enums and the DLL exports though.
Usage is all tied up in the C# scripts in the Unity package, but it seems pretty straight forward.
@n00854180t Nice! I was wondering if anyone ever going to implement OVR LipSync into UE4 
@ So, not a chance for Android version with full BP exposure?
Thanks
The Android port is coming, i have just been having some difficulties with Android packaging and how the libraries I am building link (read: winging it).
Baring some unforeseen events, I should hopefully have something available after the weekend
Unfortunately, I have been flat-out with work and life.
As I don’t work as a game dev, and this just slots in when I have some free time.
Sweet!
Totally understandable. Currently I am in the same boat, so to speak.
Thank you ! That fixed my build problems 
Hi, first of all thanks for the Plugin, it is awesome.
I have a problem and hope somebody can help me out.
I need to access the exact point in time when the microphone detects any input.
I figured it has to be the variable “Utt_started” in the script “SpeechrecognitionWorker.cpp”, but how can I turn this variable into a node in a blueprint? Tried several thing but none of them worked out.
@BigVulpes - the best way to do it would be to copy the existing WordSpoken event stuff then rename it to your own name (like UtteranceStarted) and set it to fire off when utt_started is set to true (it’s just a bool if I remember right).
Progress on the Oculus Lip Sync lib port. It’s not done yet but all the boilerplate plugin code is set up, now just have to implement passing the audio data and firing the event, and test it out.
Having voice input coupled with OVRLipSync would be crazy insanely awesome in Gear VR !!
Thanks for the quick reply.
I should have said that i am a beginner in c++
However i did rename the Wordspoken event (see pictures). But if I try to recompile the plugin in ue4 i get error messages (see pictures). Can you point me to where i did something wrong?
@BigVulpes - Instead of just renaming them, copy each function first, then rename it, but leave the original WordSpoken stuff intact. The idea is to make an entirely new event to fire off.
You’ll need to copy the relevant bits in both the .h (declarations) and .cpp (definitions).
If you’re still having problems I’ll give an example.
@ - BTW, is there a specific reason you’re using the Sphinx provided methods (through ALSA) of getting at the microphone data (other than it being convenient/and the way that’s used in the examples)?
I was looking into it and was initially going to try and use those as well but figured I’d look into getting access to the microphone samples via Unreal’s API, and found this: https://quoteunquotestudio.com/2015/02/22/microphone-input-in-ue4/
I haven’t tested the OVR Lip Sync plugin at all yet but it’s using that method and is just about ready.
https://github.com/ChairGraveyard/ovrlipsync-ue4
The lastest/should work version is on the Github repo now, btw. Again, totally untested!
I’ll be testing it this weekend and updating it with an example using the mesh that Oculus provided.