Hi everyone and sorry for the late replies!
@jackchen03281992 :
We already support some of the consoles (everything WIP). Please checkout your Engine/Platforms folder to see on which platforms the plugin is enabled (You will need to be a dev for that platform).
@grabthar :
In NNE (caution, NNE != NNI, NNI will be removed in 5.3), dynamic shapes are supported in some runtimes already (e.g. ORT cpu).
When you have loaded a model, you can access the tensor descriptors which return symbolic tensor shapes, containing a -1 for dimensions which are dynamic.
Before the first inference (and every time your input shape changes), you must call SetInputTensorShapes with the actual size of your input tensor (in specific, -1 replaced by the actual size), so that the model can run shape inference and potentially allocate the right amount of memory for intermediate tensors.
@FallenNinja :
As pointed out by grabthar, you always need the shape (describing the size of the actual input). Text input would probably be a 1D input tensor. Unfortunately, NNE currently only supports floats as in and output types, thus text is not yet supported.
But we are working on it, please stay tuned for updates on this!
@dchamploo :
Indeed, mobile is not supported yet, but we certainly want to extend device support in the future, please stay tuned!
@matt :
- Yes, NNI will be gone soon and NNE is the way to go
- Yes, NNE is currently making heavy use (but it is not limited to) the ONNX runtime. Please checkout my answer to jackchen03281992 for platform support .
I hope that helped, let me know if you have further questions!