Attach to component not animating

I’m trying to attach in these pictures a set of leg covers via attachtocomponent. For some reason when I do this the pants seem to animate correctly. They just move entirely with the character.

However, if I use set skeletal mesh it works perfectly fine except because of the way the system is setup we need to actually use an actor and not a skeletal mesh.

Most things I’ve seen point back to the master pose component, but that seems to have little effect when I’m going the first route.