Here is a simple and short solution to this problem. Simply update character movement component at every frame in Tick function:
FHitResult OutHit;
GetCharacterMovement()->SafeMoveUpdatedComponent(FVector(0.f, 0.f, 0.01f), GetActorRotation(), true, OutHit);
GetCharacterMovement()->SafeMoveUpdatedComponent(FVector(0.f, 0.f, -0.01f), GetActorRotation(), true, OutHit);
Credit for this solution goes to people in this post: character collision not working when stood still, works when moving - #6 by Ceteri