You want to look into something called Navmesh Invokers.
This requires switching Navmesh Generation to dynamic, and also ticking “Generate Navigation Only Around Navigation Invokers” in the project settings.
Then you’ll need to add a Navigation Invoker component in the AI’s blueprint.
I can’t talk to the performance of this on an infinite world or with a hell of a lot of invokers but that should hopefully give you a push in the right direction for what you want.