I’ve built a local Llama model and developed an API for it. Now, I need to integrate it into Unreal Engine 5 to create a custom offline chatbot for in-game interactions.
Has anyone done something similar or have tips for seamless integration and optimizing performance?
I’m using the Universal Offline LLM plugin. I’ve only been able to get Llama-2 to work so far. Currently, it’s unacceptably slow. However, it’s going to end up running on a HPC cluster and be pixel streamed. I hope the plugin gets updated to work with Llama-3.3.