I apologize, I left out a step: you also need to SetMovementMode for CharacterMovement component to Falling or Walking on Begin Play. reason for this is controller is what sets Movement Mode which dictates how CharacterMovement component handles physics. developers are considering initiating SetMovementMode by default when bRunPhysicsWithNoController is enabled (UE-6143).
In meantime, either that method or one mentioned by RhythmScript should get you what you need. Let me know if you have further questions!