Not stupid at all!
And the reason it was worse with complex as simple is that collision was too thin.
As the collision became thicker the issue started occurring less.
Thats what made me think it had to be character movement related - the visual of the collision was really just the confirmation.
Glad you sorted it.
You can likely improve on the IK so that it doesnt overshoot collision (assuming you made it, if thats part of the engine now, well then you need to file a bug report on it?)