Attachments Bug

so the problem is I set a child actor on my weapon actor for the attachments but it affects all the actors in the world how would i only be able to affect one of those actors (the one in the players hand)?