Runtime Speech Recognizer - Multi-platform, real-time, offline speech recognition plugin. Based on Whisper OpenAI

Distribution UE Marketplace
,
1

Runtime Speech Recognizer is an open-source plugin that enables real-time, offline speech recognition. Based on Whisper OpenAI technology, particularly whisper.cpp library, and supports multiple language models pre-selected in the plugin’s settings.

GitHub link: GitHub - gtreshchev/RuntimeSpeechRecognizer: Multi-platform, real-time, offline speech recognition plugin for Unreal Engine. Based on Whisper OpenAI technology.
Marketplace link: Runtime Speech Recognizer in Code Plugins - UE Marketplace
Documentation link: Home · gtreshchev/RuntimeSpeechRecognizer Wiki · GitHub

Features

  • Fast recognition speed
  • English-only and multilingual models available, with multilingual supporting 100 languages
  • Different model sizes available (from 75 Mb to 2.9 Gb)
  • Automatic download of language models in the Editor
  • Optional translation of recognized speech to English
  • Customizable properties
  • Easy selection of model size and language in settings
  • No static libraries or external dependencies
  • Cross-platform compatibility
2

Hello,

I’m attempting to utilize the Runtime Speech Recognizer plugin in Unreal 5.3.2. I am getting a [Blank Audio] message. I have connected the Get Sample Rate return value to the appropriate pin on the Process Audio Data node, as described here:

Unreal Engine 5.3 crash · Issue #17 · gtreshchev/RuntimeSpeechRecognizer · GitHub

Still no luck, I’m afraid.

I am sure that my mic signal is arriving, as I’ve made an input level meter widget to show it. I did this using the built-in Audio Capture component, which I added to the Blueprint while troubleshooting.

I’m hopeful that someone can provide suggestions/feedback, especially if you’ve encountered this problem.

Thanks everyone!

Ben

[UPDATE]

Strangely, it works correctly in 5.2.1, but not in 5.3.2. I will try in in 5.4.1 tomorrow and update this again.

4

Hi, sorry for the delay, I don’t check the UE forum regularly. There was this bug in the past, but as long as your setup is correct, you shouldn’t have this issue anymore in the last plugin version.

5

It works fine in UE 5.2.1 for me. I don’t know what could be different. It’s been updated a few times since I’ve last tried it in 5.4, but then again 5.4.4 just dropped today. :slight_smile: