Inference with a custom Onnx policy using the AI Agents Plugin

Hi @Deathcalibur, I wonder if the new “bring your own model” functionality introduced in the Agents plugin 5.5 version would allow inference with an Onnx model for the policy by loading from the file system. If so, would you mind clarifying the steps?

To note, I am after doing that from the Agents plugin as it could be packaged with the game as far as I understand, compared to other Unreal options that would only run using the editor without the ability to package with the game.

Thanks,

2 Likes

I can write up the steps to get it working, but it might be best for me to focus on making it easier to do directly for 5.6. I was able to get ONNX working purely in game project without having to modify the library but obviously this is easier said than done. I’ll see if we can’t sneak it in for 5.6.

BTW, the existing CPU runtime absolutely can be deployed to any end user device (PC, mobile, Switch)… Just want to clear up any potential confusion there. Python is only used at editor time. At game time there is no python.

Thanks!

Hi @Deathcalibur, in regard to the question posted by @dr.shixo, I was successful in creating a package using the NNE, but it crashes on startup due to an “Assertion failed” error. I tracked it down to a null reference when creating the NNE Runtime object on line 97 (see screenshot). In the editor Runtime is populated successfully but in the package, it fails to be created. Do you know of any way to fix that?

1 Like

I’ll try to take a look at this next week. Thanks for investigating!

Let me know if this chain of messages meets your needs or not!

1 Like

Ok, I will see if I can get this working. Thank you for your help!

1 Like