Do I need to call Destory method to remove components from Actor when Destory method to remove Actor?

You can try to force garbage collection but you should try object pooling first.

Please refer to this discussion: Force Garbage Collection