I am planning to conduct research on reinforcement learning and need to gather information and create a simple demo. I am currently planning to use ‘Learning Agents’ and would like to know how to apply a custom model. It seems that I have to modify python file in learning agents but I’m not sure. Is there anyone knowledgeable or able to provide some tips on this?
For example, I assume that I want to add cumstom layers to my policy network. I found that in train_ppo.py there is a line defining policy netowrk using class NeuralNetwork which is defined in nne_runtime_basic_cpu_pytorch.py. Then it seems that policy network is updated to trainer using ‘recv_policy’ function. Then if I want to add some layer, can I just add layer to policy network after respone = trainer.recv_policy(policy_network) line? If not, what should I do? If so, what do I have to do next? I’m sorry that I’m not familiar using Unreal Engine with python files.
Yes, essentially you would have to modify the python code. You would need to make corresponding changes on the C++ side as well or code in a way to use ONNX. This is because the model is evaluated in UE for doing rollouts in the environment (we don’t receive actions from the python side, only model weights). This is a bit inconvenient for researchers but it makes LA a much easier transition from development to deployment for game developers.
Eventually I plan to have a “bring your own algorithm” functionality (working on it now actually) but not sure when it will land. Perhaps part of UE 5.5 but no guarantee on that.
Brendan
Don’t you also have to edit the python code to adjust the number of epochs to run in training?