Learning Agents Driving Tutorial + Obstacle avoidance

[OptimusSkooch]

Hi

I’m trying to make the driving agents also avoid obstacles on the track, but I can’t get it to work. After reading all the posts about this topic, such as: Learning Agents 5.4 Question, adding obstacles to the track of the tutorial, What is the best way to add ray cast to observations? Learning Agents 5.4 and Ray cast Observation for obstacles detection and Rewards, my implementation looks like this:

Interactor

1. Specify Agent Observation
   

   

2. Gather Agent Observation
   

   
   
   
   

I basically shoot 7 rays, make array represents the angles.

3. Actions
   I didn’t change the actions.
   
   

Trainer

1. Gather Agent Reward
   I still want the car to drive forward, so I let the following rewards from the tutorial in there:
   
   
   
   
   
   n

And those are the rewards I added:
First I raycast 3 times:

Then I give a penalty if the actor gets too close to the Hit positions of the obstacles

If the car actually hits something (OnActorBeginOverlap), I also give a penalty:

I’ve experimented with different Distance Thresholds and Reward Scales, but at the end they drive as fast as they can without accounting for any obstacles.

2. Gather Agent Completion
   If they drive out of the track:
   
   
   
   If they hit something, or are about to hit something
   
   

Then I add those together with Or node

This is the track, the white dots are the obstacles:

I let this run overnight, and the result was them just hitting any obstacle on the path, not even trying to avoid. Any idea what I’m doing wrong? Any help is appreciated. Thank you!

[AlexarassK]

Maybe you’ve already checked but make sure that the basics are all working. Check that your obstacles have query collisions enabled. If they do, make sure that your collisions are hitting etc.

I am seeing something odd though. After making sure of the above and anything else you might have missed or assumed to be a fact without checking, do check that “Make proportion along ray observation” node. I have a suspicion that if you debug draw the line trace it won’t be correct. I think this function expects a local start and local finish not a world. The reason I think that is because it’s taking in a transform. Try placing the start at 0,0,0 and the end at your mesh’s forward vector multiplied by the length of your car’s vision.

[OptimusSkooch]

Thank you for your reply @AlexarassK. Make Proportion Along Ray Observation was indeed the problem. The rays were being debug drawn at wrong locations.
This solved it:

Thank you so much!

[Deathcalibur (Brendan Mulcahy)]

Awesome. You don’t know how happy it makes me to see people helping each other in the community with this. I never wanted to be the bottleneck so kudos to you both for helping each other. Thanks!

[OptimusSkooch]

Thanks a lot for the awesome plugin. Did my graduation work using Learning Agents, and it was a ton of fun!

For anyone interested: https://github.com/hovo-hakobyan/Autonomous-Driving-Learning-Agents-5.4

[Deathcalibur (Brendan Mulcahy)]

Awesome work! Just gave it a quick skim and looks interesting - will have to take a deeper look soon. Thanks for sharing!