In conclusion, the current situation does not seem to allow NNE to run a large model.
I followed the tutorial on running ONNX with NNE in UE5.3.
https://dev.epicgames.com/community/learning/courses/e7w/unreal-engine-neural-network-engine-nne/G3rx/unreal-engine-nne-overview-5-3
I was able to get it to work correctly on a small model.
However, ONNX files are limited to 2 GB.
Models larger than 2 GB, such as LLM, are split into external data files (.onnx_data).
According to the ONNX documentation, external data will be loaded if it is in the same directory as the model.
https://onnx.ai/onnx/repo-docs/PythonAPIOverview.html
There was no way to import such a split model into UE, and placing external data in the same directory as the model only resulted in the following error
LogNNE: Error: Exception during initialization: D:\GitRepos\onnxruntime\onnxruntime\core\optimizer\optimizer_execution_frame.cc:78 onnxruntime::OptimizerExecutionFrame::Info::Info [ONNXRuntimeError] : 1 : FAIL : GetFileLength for model.onnx_data failed:open file model.onnx_data fail, errcode = 2 - ?w?肳?ꂽ?t?@?C??????????????
I tried both locations where the external data was placed, where the .onnx was and where the .uasset was after importing, but the results were the same.
Is there a way to run large ONNX models? And are there plans to support it?