We just dropped something pretty exciting, at least to me, hopefully others will find it useful and or interesting.
LLMAI, a plugin that lets you have real-time AI conversations and function calling directly in Unreal Engine 5.6 at runtime.
What’s the big deal?
-
Real-time voice chat with OpenAI (yes, you can literally talk to your game!)
-
Function calling system - AI can actually control your game, spawn objects, change settings
-
Two complete demos including an AI-controlled arcade game that responds to voice commands
-
Production-ready with full source code and comprehensive docs
Perfect for prototyping AI NPCs, voice-controlled interfaces, or just having fun conversations with your projects. The AI can even play games for you!
Check it out: LLMAI Plugin | Fab
This version works with OpenAI Realtime API with all the voices, we are working on various AI provider support, along with local hosted integrations, so stay turned!
Questions, feedback, or cool use cases welcome!
Would love to see what the community builds with this.
Cheers!