Navmesh AI pathfinding sometimes takes very strange paths

I’m working on an RTS game and have a dynamic navmesh setup. My character can build these red blocks which have an extremely high-cost navarea assigned to them. The character’s nav agent radius is way more than enough to fit through small gaps between buildings and he can easily maneuver between buildings. So usually everything is fine, but sometimes the guy randomly decides to take a detour and take a very strange path as seen in the screenshot.

In this example the green square represents his destination and I use the “AI MoveTo” node to move my guy there. Clearly the optimal path is to move directly upwards, but instead decides to go in the path as depicted by the red arrow. The weird thing is that if the guy takes the red suboptimal path to the green square, then decides to move back to his original position he might take the optimal path back.

Anyone know why my guy takes suboptimal paths at seemingly random?