Tutorial: Learning Agents Introduction

General Tutorial & Course Discussions
42

Hi Brendan!

Thanks a lot for the explanation! :slight_smile:
I tried it out, and it works.

Best,
Rudi

2 Likes
44

I have a few questions if you donā€™t mind answering:

  • I guess the official release will be once UE 5.3 officially comes out?

  • Although how is the stability and completeness of the plugin in right now UE 5.3 GitHub version? Iā€™m tempted to build UE from scratch just for this plugin.

  • Is there any community or discord around Learning Agents? It might be an interesting place to share information and tutorials.

  • Is there any ETA on when the learning course and examples might come?


Thanks.

2 Likes
45
  • I guess the official release will be once UE 5.3 officially comes out?

Yes, Learning Agents is an engine plugin that will be coming out with 5.3

  • Although how is the stability and completeness of the plugin in right now UE 5.3 GitHub version? Iā€™m tempted to build UE from scratch just for this plugin.

Learning Agents itself is stable but the rest of UE may be unstable. 5.3 has recently branched from ue5-main so you may have more stability on that branch: https://github.com/EpicGames/UnrealEngine/tree/5.3/Engine/Plugins/Experimental/LearningAgents

  • Is there any community or discord around Learning Agents? It might be an interesting place to share information and tutorials.

There is no official community outside these Forums at the time. These dev forums have somewhat recently added a ā€œLearning-Agentsā€ tag. You can search for new topics with that tag and even setup notification settings in your profile. I have notifications setup myself so I should be able to see and respond to any new topics related to Learning Agents :smiley:

Is there any ETA on when the learning course and examples might come?

The best I can offer is that they will be posted at the latest when 5.3 is released. If we get them done early, we can probably post them sooner but I canā€™t make any promises.

Thanks for the questions,
Brendan

3 Likes
46

Hi, the plugin specifically says that itā€™ll enable physically based animation. Can I implement a deep rl method, something like CALM in it, train it inside Unreal Engine using pytorch and then let ISPC do its magic?

47

Hi,

You can definitely use the plugin to try to implement basic physically based animation techniques such as DeepMimic, however right now the plugin is a little limited in terms of which network architectures it supports so you might find it difficult to re-implement something as complicated as CALM. Hopefully more complex setups such as what is used in CALM are something we will be able to support in the future though!

Thanks,

Dan

2 Likes
48

Hi Daniel,
Thanks for answering! Iā€™ve been following your work too and was pleasantly surprised to see that you have joined Epic. Hope to see more interesting stuff by you!

So i just read in the architectural details of CALM that all networks used are fully connected networks. Does that mean that their architecture is compatible?

49

Hi,

Thanks!

The difficulty is in the fact that they simultaneously train a discriminator network to help the motion be more realistic. Right now we donā€™t support training additional networks simultaneously along with the main policy. This could potentially be hacked in to the current framework. In the future it is something that would be nice to have supported properly.

2 Likes
50

Hi Dan,

Is it possible to run inference on pre-trained policies (performing a sub-task) while training the main policy? Such as the typical case of using a pre-trained ā€œlow-levelā€ controller as part of an RL workflow.

51

@Zuwa_G

Is it possible to run inference on pre-trained policies (performing a sub-task) while training the main policy? Such as the typical case of using a pre-trained ā€œlow-levelā€ controller as part of an RL workflow.

We designed Learning Agents so that it should be possible to have multiple policies with the possibility of doing what you have described.

That said, we havenā€™t had the opportunity to do extensive testing with this exact setup yet so there may be unforeseen issues that arise.

It should be an simple as training the low level policy, then using the saved weights while training the new policy.

Brendan

1 Like
52

Hi
Any update about Learning Course and Examples ?
Thanks!

1 Like
53

Thank you for this project!
I played a little bit with it and created a simple C++ demo while exploring the plugin.

Looking forward for more updates and samples!

mexxik/UE-LearningAgents (github.com)

agents

5 Likes
54

Can I simply use pytorch if this plugin doesnt yet provide a functionality? Will I be able do inference too in a shipped game if I use pytorch from the pythonfoundationpackages plugin?

55

@Mahzad
Barring any unforeseen circumstances, we will post the learning materials the week of Sept 4th.

@maxsmexx1

Awesome work! Thank you for sharing. I will give it a more detailed look when I have some time.

@user12312
Not really, no. The python portion of the plugin gets removed at compile time for Shipping builds. We will be expanding the built-in functionality of the neural network architecture with the next release of the plugin, so this will hopefully feel like less of a limitation over time.

Thanks,
Brendan

1 Like
56

Iā€™m new to all the Machine Learning stuff happening in Unreal and I know a lot is changing with every update. But maybe I could get some clarity on what the difference is between these plugins in 5.2, along with how they are different from Learning Agents

image
image1410Ɨ292 40.1 KB

Just starting to dive into Python, and hopefully will pick up on Machine Learning. End goal is to train custom cartoony characters on a mix of hand animation and mocap so they can become Chatbot AI characters with natural motion and emoting on what theyā€™re saying.

Thinking of a system like this

Or an Unreal Plugin that I can give the character mesh FBX, along this a tag system with all the animation clips to train the AI Character to move naturally, accordingly to the style and proportions of that character.

2 Likes
57

Hey Tom,

NNI/NNE are the neural network inference engine that are built into UE. Basically you can take a model trained in Python, export it to ONNX, then import and run that network in UE on NNE. NNI is the old deprecated version of NNE. More info here: NNE - Overview | Tutorial

Learning Agents is a reinforcement and imitation learning library. It allows you to gather data and train charactersā€™ behavior & decision making. Learning Agents uses neural networks under the hood, but you can also train non-neural network models and run them (if you get a bit creative). Using RL and neural networks together has been typically referred to as ā€œdeep reinforcement learningā€, which is what most cutting-edge research in RL uses today.

Currently, Learning Agents uses custom inference code written directly in ISPC, rather than using NNE. We did this to get the best performance, but the trade-off is that there is reduced flexibility. Learning Agents can only currently use feedforward, fully connected networks. We are working on expanding this functionality in subsequent releases.

I hope that answered your questions and was not too long-winded.

Thanks,
Brendan

P.S. ML Adapter is our older version of Learning Agents. ML Adapter had a completely different design philosophy that we donā€™t think will work well for games, so we decided to start fresh.

3 Likes
58

Thank you for explaining that!

I noticed on your initial post you put physics-based animations. Would this be similar to NVidiaā€™s ASE? But trained and built directly in Unreal? Because that sounds like exactly what Iā€™m looking for!

59

Yes, something we would like to cover with Learning Agents but their exact approach would be largely impossible to implement currently.

See earlier posts in this thread for similar questions about CALM.

2 Likes
60

Thank you for responding so quickly!

I see. Hmm, well I canā€™t wait for 5.3 to be fully released, and hopefully some tutorials to see just how much can get done this this new Plugin! Thanks for working on it!

61

@tomhalpin8 what do you want to do with ASE or Calm?

62

@user12312 Instead of human motions. More cartoony stylized motions. Characters with crazy proportions trained on possibly hand animation.

1 Like