For those who are going to try this solution, I ended up finding a bug, if you create a new agent it seems that the pawns that were already at the level will ignore the new navmesh. Deleting them and putting them back in solves the problem. This does not happen with pawns that are created at runtime