I know this is probably a bit random, but is there an API for handing audio input in UE4? I figured I’d maybe have a go at hacking a speech recognizer into UE4 to go with my Oculus (so I can use voice commands within the VR world), but I’ll need to grab the audio stream to pipe into the recognizer and I haven’t looked at the UE4 audio systems as yet.
I don’t suppose anyone else has already added a speech recognition/synthesis engine into UE4 have they?
There’s a guy working on Kinect v2 integration and he’s trying to add speech recognition with it.
You can try to intergrate the openair
I am very concerned about the progress of this feature. Can you tell me?
If you’re bumping a nearly 4 year old post for a piece of hardware that is no longer produced - you’re likely not going to like the answer.
I mean sheet son are you even a hacker. Just write a speech recognizer program in python and based on certain keyword or phrases send certain commands to the UE4 client using json via sockets or something and excecute commands in the VR World if you want.
You can take a look at the RuntimeSpeechRecognizer plugin, which is based on Whisper AI, namely whisper.cpp. It’s cross-platform and works offline. It’s also completely free and open source.
Marketplace link: https://www.unrealengine.com/marketplace/en-US/product/runtime-speech-recognizer