I haven’t ran into this before, but I believe the issue is that you need to ensure that the policy batch size is larger than the policy window size in these settings:
What values were you using? I can reproduce the issue and add a warning/error message in UE.
EDIT: It could also be the case that your episode has zero steps in it, which would means that you’re trying to get a random value between (0,0) which isn’t possible. So double check if perhaps that is actually the issue.