First, this typically happens when you haven’t configured the navmesh correctly for your AI character capabilities (appropriate radius, appropriate path planning, etc.) You may want to adjust the navmesh (and maybe add invisible blockers) until this doesn’t happen.
Second, you can store the last point the character was on the navmesh inside an occasional tick(), and return to that point if the character ever finds itself outside the mesh.