This appears to still be the behavior even as recent as 4.26, whether a bug or intentional.
As an additional workaround that may be simpler, disabling and enabling collision on the instanced static mesh seems to updated the collision response. I’ve only tested it with trace channels and not full collision, but expect it works to fix that as well.
