Using Chaos Anchor Fields with Sequencer

Hi All,

I am trying to use chaos destruction for buildings that are destroyed by a monster in sequencer. To keep the buildings planted on the ground, I am using anchor fields. When no anchor is added over the building, physics works as expected (well, to some extent):

When an anchor field is overlapping the building, chaos physics no longer interacts with the sequencer animation (but will to other chaos objects).

How can I allow for the sequencer animations to still interact with chaos objects when an anchor field is applied?

Anchor field blueprint below for reference: