I know this is late, but I had the same issue and I hope the next person who needs it can get it.
The fix for my near identical problem was the navmesh was calling for collision against the objects themselves. The actors I had were set to check complex collision, so I had forgotten about it. In the mesh settings of the meshes near the issues I had to go into the editor>collision>Collision complexity and change it to use complex as simple.