Also, a couple of QoL improvements for you to perhaps consider.
-
Send Message to GPT async node should have an enum to let me choose the model per async call (dynamcially changing models on the fly).
-
the node should also expose hyperparameters for the models inside an options struct, like top_p, top_k, temp, ctx_len, and so forth (popular hparams nothing too crazy), rather than in the project settings, allowing us to program dynamic user interactions with the models to their own customisability.
-
expose the initial prompt for us to prime the model outside of whatever you’ve hard-coded in the plugin. The initial prompt sets the tone of the AI and can mean a very big difference in how the nn responds. This might be okay in the Project Settings, however it would also be good as a setting in the node’s options struct as to change this on the fly after resetting/clearing the conversation history would be really handy.
-
people use stop sequences a lot also - which will stop the conversation if a word in this string array is equal.
-
error messages from the API would be handy for errors that might sit outside that which the async node callbacks don’t provide, or need more context to.
Hope I don’t come across too demanding, this is just what I observed so far while exploring your magical work. Thanks again!