Stacking NavMeshes will not work because the OP wants true 3D navigation and NavMesh is a 2D solution. Yes it works in a 3d space, but it’s not a 3d problem. My recommendation is to learn how NavMesh solves pathing and extend that into 3d by creating your own custom solution. Either way, you’re going to need to make your own pathfinding alg for now. And it might take some work, but you could leverage UE4 in certain ways…
My advice: Look up Portal Based Culling. It does scene culling by grouping scene geometry into rooms and hallways, each of which are connected by “portals”. Portals(Doors, archways, exits, entrances, etcs) in this case are simply intersections between bounding boxes (volumes). Each volume represents a room or hallway. Each room or hallway contains scene geometry. To briefly explain: If you are in a hall way, and the game tries to determine what you can see, it will assume you can see everything in that hallway (pending frustum culling). It will then cast rays to each portal and if the ray intersects only that portal it is implied that you can see beyond it into the next room or hallway, resulting in all of its objects being potentially visible. If your back is turned to a portal, or the portal is completely behind some occluder (the ray intersected something else), then you can’t see what’s in the next room so the renderer knows to ignore all objects.
The data structure used to create a portal scene hierarchy is a graph of nodes with each transition being the portals themselves, and each node being the area. This is perfect for path finding and in fact this is how NavMesh does it, except it uses planes (triangles) to represent areas, with the edge of the triangle being the transition. Same solution, but less complex.
You can easily use something like this for 3d navigation.
If you want to move toward an object…
- Determine which room or hallway the object is in.
- Do a graph traversal to determine which rooms and hallways you need to travel through to get there. (You can use a path finding algorithm of your choice for this.)
- Move toward each “portal” until you are in the same room or hallway as the object, otherwise move directly toward the object if you can see it.
How you create these rooms, hallways and portals is up to you. The first thing I’d try to do is leverage UE4s existing volume objects, and figure out how to extrapolate planes that represent the intersections between them at load time. These planes would be your portals. I would then construct my maze out of actors, with each actor being a piece of geometry that has your new custom navigation volume attached to it. That way you can create your maze either manually or dynamically by attaching these actors together. The volumes would then intersect, and from these intersections you’d create your portals.
I hope it helps.