[MENTION=193494]aloneball[/MENTION] - Thank you 
I want to thank everyone for your continuous stream of encouragement, I didn't manage to reply to all but I deeply appreciate every single message from everyone!
Re: optimizing unusued voxels, I've tried two approaches:
1) A TSet driven "fully-lazy-loaded" system that would only create voxels "on-demand". The performance was awful! Apparently hash operations for looking up a unique voxel with an identifier(x, y, z) was just too slow. Granted, my hash function was probably suspect; I might revisit this some day.
2) TArray with "reallocation-aware" usage: For best performance, voxel TArrays are only allocated once (BeginPlay) and references/pointers to individual elements are used by the various caches. This is how the system works presently.
Now, to allow for "infinite worlds" where only those voxels we truly need are maintained the first step I took was to eliminate all dependencies on references/pointers in the code as these would be invalidated each time the TArray gets realloacted. So when I substituted all references with a tiny struct to hold a voxel's unique identifiers (x, y, z) and tested it, performance dropped significantly. The plan was shelved right there and I didn't even dare to try expanding the TArray on demand to see what the performance was like; I suspect it would have been even slower.
One idea brewing in me now is a "nominal world" that gradually expands each tick as the game goes on, eventually maturing to "full world size". Won't work for all games (especially if players/AI can teleport rapidly) but it's something. I don't think serializing voxels to disk will help load times though. AFAIK rehydrating from disk into struct objects is slower than just creating them from scratch on BeginPlay.
N.B. Collision checks are never performed on BeginPlay anyway, they're always lazy-loaded. If you turn debug voxels on with bAutoInitializeVolumes to false, the whole world will be full of green voxels until you actually start running navigation queries! Voxel collision for static meshes could be useful to cook, but this is not the real bottleneck for load times right now - the mere act of constructing millions of empty voxels whether loaded from disk or from code, is the thing determining load times now.

I want to thank everyone for your continuous stream of encouragement, I didn't manage to reply to all but I deeply appreciate every single message from everyone!
Originally posted by MatzeOGH
View Post
1) A TSet driven "fully-lazy-loaded" system that would only create voxels "on-demand". The performance was awful! Apparently hash operations for looking up a unique voxel with an identifier(x, y, z) was just too slow. Granted, my hash function was probably suspect; I might revisit this some day.
2) TArray with "reallocation-aware" usage: For best performance, voxel TArrays are only allocated once (BeginPlay) and references/pointers to individual elements are used by the various caches. This is how the system works presently.
Now, to allow for "infinite worlds" where only those voxels we truly need are maintained the first step I took was to eliminate all dependencies on references/pointers in the code as these would be invalidated each time the TArray gets realloacted. So when I substituted all references with a tiny struct to hold a voxel's unique identifiers (x, y, z) and tested it, performance dropped significantly. The plan was shelved right there and I didn't even dare to try expanding the TArray on demand to see what the performance was like; I suspect it would have been even slower.
One idea brewing in me now is a "nominal world" that gradually expands each tick as the game goes on, eventually maturing to "full world size". Won't work for all games (especially if players/AI can teleport rapidly) but it's something. I don't think serializing voxels to disk will help load times though. AFAIK rehydrating from disk into struct objects is slower than just creating them from scratch on BeginPlay.
N.B. Collision checks are never performed on BeginPlay anyway, they're always lazy-loaded. If you turn debug voxels on with bAutoInitializeVolumes to false, the whole world will be full of green voxels until you actually start running navigation queries! Voxel collision for static meshes could be useful to cook, but this is not the real bottleneck for load times right now - the mere act of constructing millions of empty voxels whether loaded from disk or from code, is the thing determining load times now.
Comment