SetCollisionResponseToChannels not working unless clicked in editor

I encountered the same issue where setting the response for a specific channel did not take effect unless set in the editor. I solved it by re-creating the physics state for the Primitive Component (Primitive->RecreatePhysicsState()).