LearningAgents: FEpisodeBuffer::GetRewards() using incorrect index?

Development Programming & Scripting
, ,
1

Hi,

While upgrading our version of the engine to 5.5.4 I was reading through the LearningAgents code and noticed FEpisodeBuffer::GetRewards() seems to use the incorrect index for fetching rewards, where it does RewardArrays[RewardId][RewardId] instead of RewardArrays[RewardId][InstanceIdx]. Notice that PushRewards() uses RewardArrays[RewardId][InstanceIdx]. Also the other Get/Push functions use the InstanceIdx in the same way PushRewards() does, making this more suspicious.

`const TLearningArrayView<2, const float> FEpisodeBuffer::GetRewards(const int32 RewardId, const int32 InstanceIdx) const
{
UE_LEARNING_CHECKF(RewardId >= 0 && RewardId < RewardArrays.Num(), TEXT(“Reward id invalid!”));

// vvvvvvvv Should be InstanceIdx?
return RewardArrays[RewardId][RewardId].Slice(0, EpisodeStepNums[InstanceIdx]);
}`

As an aside, the whole reason we are doing this is to be able to calculate the same experience stats that the python script does - specifically the avg_reward and avg_reward_sum. Ideally this calculation would be made in C++ and passed to python so we could retrieve it instead of attempting to redo the calculation ourselves.

Thanks!

2

Great catch. I will push up the fix shortly.

3

Thanks for the quick turnaround. Can you paste the fix git sha or at least part of the commit message? We use git so it’ll make it easier to find.

4

No problem, thank you!

5

For future reference, it looks like the sha is 7eb4d365154f15fab5de21050d4eef93e4c78bed

https://github.com/EpicGames/UnrealEngine/commit/7eb4d365154f15fab5de21050d4eef93e4c78bed

6

“Learning Agents: Fix a bug with GetRewards” on ue5-main. I don’t know the git sha. It looks like it hasn’t been pushed up to GitHub yet.

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.