Avoid navmesh rebuild lag?

Hey there! This is my first post, so I will try my best to be clear and as little of a pain as possible.

Alright so the scenario:

I am creating a turn-based top down game that is not using a grid. Everything is going great, except for one problem. During combat, I have the ai turn off their “can ever effect navigation” so that they do not stop themselves from pathing. Upon ending their turn, I have them turn it back on to create an obstacle so that other ai/player can avoid them while moving around. I -would- use something like RVO avoidance, but I need them to stick to their navpath which I am finding synchronously and creating a visual spline that matches it. The distance traveled along this spline determines action points spent, so therefore they have to stick to it.

This all works well and good, until they turn their navmesh affecting back on. Not every single time, but often enough it causes a momentary chug in the game, dropping sometimes from 120 fps all the way to the teens. This chug lasts typically between a quarter-second all the way up to 2 seconds. (seems random)

Is there a way that I can continue using the navmesh to help create my spline paths, whilst avoiding this performance hitch that about 75% of the time appears?

One thought I literally just had was maybe it’s because I’m running the project off an external hard drive, but I kind of doubt that’s the case if most of the time it’s running at at least 60 fps. (usually around double that)

Here’s a video showing the issue. (Of course when I record, it’s not as bad, but it’s still visible at around 1:15ish)