After some more testing, it looks like the overlaps seem to work and the remove instance is triggered, but the mesh does not disappear in-game
It does work when I have the overlap event happening on my character:
