NPU support: is currently very limited, defined on what is accessible through DirectML. It is basically limted to intel for now, as reported here. But with this being steadily improved, I am confident we will soon get wider support.
If you are interested in SNPE, you should definitively have a look at Qualcomm’s runtime here. I never used it myself, but Qualcomm did a great job in opening up NNE for mobile.
NNERuntimeORT upgrade: Yes, we try to increase the upgrade cadence to have more recent versions in future releases. Apologies for the delays.
The RDG interface: It will not be as easy as the GPU or CPU interface. The CPU and GPU interface are very close to what people are used from training environments like torch or TF. To do in-frame inference you need to know a little more details about Unreal, as it is very engine specific. In the short: The inference call EnqueueRDG will need you to pass a Render Dependency Grah builder, which you have access to e.g. when you add your own ViewExtensionBase class for e.g. post processing. So the interface is more powerful, as you run inference on frame resources without any CPU sync, but it needs a little more knowledge about UE
Hi @ranierin, is there a list of the supported platforms for NNE runtimes? I saw the message below in the NNE overview docs, but it doesn’t link to the supported platforms. For example, I could run ORTCpu on Win64 within the editor without issues, but when packaging with Win64 as the target platform, I got the Null pointer for runtime!
Not all runtimes are available on all platforms, even when the corresponding plugin is enabled. If a runtime is not available, or if it is available but does not implement the interface passed in the templated function, the returned weak pointer will be null. As runtimes can unload themselves, you should run a test for validity of the weak pointer before using it.
Runtimes typically register, unregister, load, and unload themselves along with their related plugin and module. However, runtime’s lifetime and registration is up to its specific implementation.
In the below code, if I run this in the editor, it works fine. However, when I run a package it fails because Runtime (created on line 97) is null. This is on UE 5.4
Hi @dr.shixo , there is no list, but NNERuntimeORTCpu should definitively also work on Win64 packaged builds. Which engine version are you using? Can you check if the NNERuntimeORT plugin is loaded properly in the packaged build? It is the first time we hear of someone having issues with this.
