How can I run Learning Agents inference on GPU? Is it possible to switch the NNE runtime provider through Blueprints, or does this require a C++ implementation?

Development Programming & Scripting AI
, ,
1

How can I run Learning Agents inference on GPU? Is it possible to switch the NNE runtime provider through Blueprints, or does this require a C++ implementation?

2

You can’t very easily. Inference has to be done on the CPU at this time.

3

Thanks Brendan.
I thought of giving it a try and have made the required changes in LearningNeuralNetwork.cpp. However, I’m getting the following error during inference:
[2025.08.31-20.08.20:604][400]LogNNERuntimeORT: Error: ONNX Runtime error 7: Failed to load model because protobuf parsing failed.

The error seems to originate from:
CreateOrtSessionFromArray() called from
ModelInstances_GPU.Emplace(InModel.CreateModelInstanceGPU());
inside
void FNeuralNetworkInference::ReloadModelInstances(NNE::IModelGPU& InModel, const int32 InInputSize, const int32 InOutputSize) {}
which I created for the GPU model.

Since I am loading the same (CPU) model with ModelData->Init(TEXT(“onnx”), FileData); modified onnx instead of ubnne. I assume this error is expected.

Do you think saving the model in the correct format during training should fix this? If yes, can you guide me little. I’m not concerned about performance for now.