Why sequencer keeps interferring with my actors?

How do I prevent the sequencer from possessing the object when it is not active?