Excessive Overlap/EndOverlap Calls within a short span of time

I have an issue where an enemy is overlapping with my player in a certain situation multiple times within a short span of time.

Basically, I have an enemy with the default cylinder collider, and I added a capsule to it, to generate overlap events with my player.
This works and events are generated, but when I am moving exactly out of the enemy’s capsule (somewhere on the edge of it) I get bombed with tons of OnOverlap and OnEndOverlap.
I feel like I am missing something basic to prevent it from happening (tried to debounce the overlap logic but it all seemed wrong to do).

Here’s some code from my enemy class:

in BeginPlay:

	DamageCapsule->OnComponentBeginOverlap.AddDynamic(this, &ADDUEnemy::OnCapsuleOverlap);
	DamageCapsule->OnComponentEndOverlap.AddDynamic(this, &ADDUEnemy::OnCapsuleEndOverlap);

OnCapsuleOverlap:

	if (HasAuthority() && OtherActor != this && !OtherActor->ActorHasTag("Enemy") && !HitActors.Contains(OtherActor))
	{
		if (UAbilitySystemComponent* TargetASC = UAbilitySystemBlueprintLibrary::GetAbilitySystemComponent(OtherActor))
		{
			HitActors.Add(OtherActor);
			DamageEffectParams.TargetASC = TargetASC;
			const FActiveGameplayEffectHandle EffectHandle = UDDUAbilitySystemLibrary::ApplyDamageEffect(DamageEffectParams);
			ActiveEffectHandles.Add(OtherActor, EffectHandle);
		}
	}

OnCapsuleEndOverlap:

	if (HasAuthority() && OtherActor != this && ActiveEffectHandles.Contains(OtherActor))
	{
		if (UAbilitySystemComponent* TargetASC = UAbilitySystemBlueprintLibrary::GetAbilitySystemComponent(OtherActor))
		{
			if (const FActiveGameplayEffectHandle* EffectHandle = ActiveEffectHandles.Find(OtherActor))
			{
				TargetASC->RemoveActiveGameplayEffect(*EffectHandle);
				ActiveEffectHandles.Remove(OtherActor);
			}
		}
		HitActors.Remove(OtherActor);
	}

Whenever the DamageCapsule overlaps, I apply a periodic GE, and I remove it on EndOverlap. not sure if it has anything to do with it.

The DamageCapsule on my enemy is WorldDynamic, and set to overlap with a Pawn.
and in my player blueprint, the cylinder default collider is a Pawn, and set to block WorldDynamic.

Any hints on how to solve it?