Runtime Speech Recognizer - Multi-platform, real-time, offline speech recognition plugin. Based on Whisper OpenAI

Distribution UE Marketplace
,
1

Transform your game with real-time, offline, cross-platform speech recognition powered by OpenAI Whisper technology!

Experience fast cross-platform speech recognition that works completely offline, featuring GPU acceleration on Windows and optimized CPU processing on other platforms. Integrate voice-driven menus, hands-free gameplay commands, or specialized assistive features with ease.

Quick links:

Key features:

  • Real-time Recognition: Convert speech to text instantly as your players speak
  • Offline Operation: No internet connection required - everything runs locally
  • High Performance:
    • Windows: Vulkan-powered GPU acceleration
    • Other Platforms: Optimized CPU processing with intrinsics
  • Cross-Platform: Works seamlessly across all platforms
  • Multi-Language Support: Recognize and translate 95+ languages
  • Flexible Integration: Works with any audio input source
  • Voice Activity and Command Detection: Smart noise filtering and command detection for accurate results
  • Customizable Parameters: Fine-tune the recognition process for your needs
2

Hello,

I’m attempting to utilize the Runtime Speech Recognizer plugin in Unreal 5.3.2. I am getting a [Blank Audio] message. I have connected the Get Sample Rate return value to the appropriate pin on the Process Audio Data node, as described here:

Unreal Engine 5.3 crash · Issue #17 · gtreshchev/RuntimeSpeechRecognizer · GitHub

Still no luck, I’m afraid.

I am sure that my mic signal is arriving, as I’ve made an input level meter widget to show it. I did this using the built-in Audio Capture component, which I added to the Blueprint while troubleshooting.

I’m hopeful that someone can provide suggestions/feedback, especially if you’ve encountered this problem.

Thanks everyone!

Ben

[UPDATE]

Strangely, it works correctly in 5.2.1, but not in 5.3.2. I will try in in 5.4.1 tomorrow and update this again.

4

Hi, sorry for the delay, I don’t check the UE forum regularly. There was this bug in the past, but as long as your setup is correct, you shouldn’t have this issue anymore in the last plugin version.

1 Like
5

It works fine in UE 5.2.1 for me. I don’t know what could be different. It’s been updated a few times since I’ve last tried it in 5.4, but then again 5.4.4 just dropped today. :slight_smile:

6

(post deleted by author)

7

I almost gave up. No matter what I did, I’d either get “You” or “[BLANK AUDIO]” print strings. Then it occurred to me that I recently installed NVIDIA Broadcast on all my computers, and gave it access to my microphones (built-in and external). So, I made a simple integer variable and plugged it into the Device Id pin of the Start Capture node. I compiled, and then changed the default value from 0 to 1. Bingo! Now that I’m aware of this, I think I’d better come up with a UI to allow players to select their mic input source. In case it matters, I also toggled VAD on just before Start Capture. I hope this helps anyone who might be experiencing similar frustration!

1 Like