I don’t think you can entirely remove the collision detection, because the character movement component needs it to stay on the ground and not walk through trees and such.
That being said – maybe the problem is that you have too many trees and such? Are they part of the walkable area? If you remove collision meshes from your trees, and instead just put large collider boxes around those areas, does the frame rate improve?
It’s hard to tell what else could be going on without a real performance deep-dive, though. Hence, “you should pair or contract with someone experienced if you want to go forward with this :-D”