# Runtime Local LLM (llama.cpp)

**URL:** <https://forums.unrealengine.com/t/runtime-local-llm-llama-cpp/2718296>\
**Category:** Product Discussion\
**Tags:** fab\
**Created:** [April 27, 2026, 1:04pm UTC](https://forums.unrealengine.com/t/runtime-local-llm-llama-cpp/2718296 "2026-04-27T13:04:54Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![gtreshchev](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/gtreshchev/32/1075419_2.png) [@gtreshchev](https://forums.unrealengine.com/u/gtreshchev)\
**Post date:** [April 27, 2026, 1:04pm UTC](https://forums.unrealengine.com/t/runtime-local-llm-llama-cpp/2718296/1 "2026-04-27T13:04:54Z")

</div>

## **🧠 Run large language models entirely on-device in Unreal Engine - offline, cross-platform, powered by llama.cpp**

Run **GGUF-format LLMs** (Llama, Mistral, Phi, Gemma, Qwen, TinyLlama, and more) directly within your Unreal Engine project with **no internet connection, no API keys, and no cloud dependencies** at runtime. The plugin wraps [**llama.cpp**](https://github.com/ggml-org/llama.cpp) with a full Blueprint and C++ API, load models, send messages, and receive token-by-token streamed responses, all on a background thread with game-thread callbacks.

### **Quick links:**

- 🎮 [**Packaged Demo Project (Windows)**](https://georgy.dev/runtime-local-llm-demo-windows)
- 📄 [**Documentation**](https://docs.georgy.dev/runtime-local-llm/overview)
- 🎥 [**YouTube video demonstration**](https://www.youtube.com/watch?v=lVvODaHntBE)
- 💬 **Discord support chat**
- 📌 **Plugin Support & Custom Development: solutions@georgy.dev** (tailored solutions for teams & organizations)

### **Key features:**

#### **🎯 Core Capabilities:**

- **Complete offline inference:** no cloud services or subscriptions required
- **GGUF model support:** load any GGUF-format model (Llama, Mistral, Phi, Gemma, Qwen, TinyLlama, etc.)
- **Up-to-date llama.cpp:** updated regularly on Fab to keep pace with llama.cpp releases, so the latest GGUF model formats are always supported
- **GPU acceleration:** Vulkan on Windows and Linux, Metal on Mac and iOS, optimized CPU + intrinsics on Android and Meta Quest
- **Cross-platform** : **Windows, Mac, Linux, Android** (including **Meta Quest** ), **iOS**

#### **⚡ Model Loading & Management:**

- [**Load by model name**](https://docs.georgy.dev/runtime-local-llm/how-to-use-the-plugin#load-by-name) with a dropdown selector in Blueprints
- [**Load from local file path**](https://docs.georgy.dev/runtime-local-llm/how-to-use-the-plugin#load-from-file-path)
- [**Download from URL and load automatically**](https://docs.georgy.dev/runtime-local-llm/how-to-use-the-plugin#load-from-url): skips download if the model already exists on disk
- [**Download-only mode**](https://docs.georgy.dev/runtime-local-llm/how-to-use-the-plugin#download-models-at-runtime) for pre-caching models (e.g. on a loading screen or settings menu)
- [**Editor model manager**](https://docs.georgy.dev/runtime-local-llm/managing-models-in-the-editor): browse a built-in catalog, download, import custom GGUF files, delete, and [**test models directly in project settings**](https://docs.georgy.dev/runtime-local-llm/managing-models-in-the-editor#testing-models-in-editor)

#### **🗣 Inference & Conversation:**

- [**Token-by-token streaming**](https://docs.georgy.dev/runtime-local-llm/how-to-use-the-plugin#send-messages): receive each token as it generates for real-time display
- [**Configurable inference parameters**](https://docs.georgy.dev/runtime-local-llm/inference-parameters): temperature, Top-P, Top-K, repeat penalty, GPU layer offloading, context size, seed, thread count, and system prompt
- **[Conversation management](https://docs.georgy.dev/runtime-local-llm/how-to-use-the-plugin#reset-context)**: multi-turn conversations with context reset, save/load to disk, in-memory snapshots, and automatic summarization for long-running chats
- [**Per-message system prompt override**](https://docs.georgy.dev/runtime-local-llm/how-to-use-the-plugin#send-messages)
- [**Generation cancellation**](https://docs.georgy.dev/runtime-local-llm/how-to-use-the-plugin#stop-generation) at any time

#### **🛠 Development Features:**

- Full **Blueprint** and **C++** API with async nodes and delegate-based callbacks
- [**Model library functions**](https://docs.georgy.dev/runtime-local-llm/how-to-use-the-plugin#model-lib-functions) for querying available models, checking disk presence, retrieving metadata
- [**Automatic packaging**](https://docs.georgy.dev/runtime-local-llm/managing-models-in-the-editor#packaging-settings): models ship with your project via NonUFS staging with no manual configuration
- Comprehensive [**error handling**](https://docs.georgy.dev/runtime-local-llm/how-to-use-the-plugin#error-codes) with descriptive error codes

#### **🎮 Perfect for:**

- NPC dialogue and dynamic conversations
- In-game AI assistants and companions
- Procedural content generation (quests, lore, item descriptions)
- Voice-driven gameplay workflows (paired with [**Runtime Speech Recognizer**](https://fab.com/s/695da4f72d87) and [**Runtime Text To Speech**](https://fab.com/s/944777c6f7ee))
- Offline chatbot interfaces
- Educational and training applications
- Privacy-sensitive deployments with no data leaving the device

#### **🌟 Compatible plugins:**

- [**Runtime AI Chatbot Integrator**](https://fab.com/s/08d6e9e15cd1): cloud-based LLM APIs (OpenAI, etc)
- [**Runtime Text To Speech**](https://fab.com/s/944777c6f7ee): offline TTS for speaking LLM responses
- [**Runtime Speech Recognizer**](https://fab.com/s/695da4f72d87): offline speech-to-text for voice input
- [**Runtime MetaHuman Lip Sync**](https://fab.com/s/f01459d96a81): real-time lip sync driven by TTS output
- [**Runtime Audio Importer**](https://fab.com/s/f01459d96a81): runtime audio processing and playback

> **[Runtime Local LLM](https://www.fab.com/listings/47d00943-f8ef-4acf-9861-68a51d478766)**
>
> 🧠 Run large language models entirely on-device in Unreal Engine - for AI-driven NPCs, dynamic dialogues, and offline chatbots. Cross-platform, powered by llama.cppRun GGUF-format LLMs (Llama, Mistral, Phi, Gemma, Qwen, TinyLlama, and more) directly...
