A while ago, when I was working with editor, midsession, a message would popup, telling me zen is not available and would instantly crash the editor, losing hours of work. I then disabled zen and midsession crashes stopped.
Later after update 5.6.4 (probably 5.6.3) I came across a situation that on random days, editor would randomly refuse to load instantly and would start compiling shaders again, delete DDC cache, or crash on start up telling me the cache does not have a writable nodes. All this is happening while the DDC cache is in same drive as engine and project.
As usual I spend more time figuring out bugs and fixing bugs in Dev Environment than actually producing some outcome. so I go through documents, logs … to find out what’s wrong.
The document Using derived data cache in unreal engine says the following:
How the DDC is Structured
- When an asset needs derived data, your system checks the fastest cache first, then the next fastest, and so on, until it finds the data.
- When the data is found, your system copies it into the fastest local cache so it can access it more quickly next time.
- If your system does not find the data, it generates new derived data for the asset, then asynchronously copies it into the caches so it is available for you and your team in the future.
This is a bad idea, since
- You are not allowing the user to configure their fastest storage themselves. Give credit to user and accept that probably user knows which medium is faster and can order the cache locations themselves.
- You choose the fastest device based on wrong telemetry, in my system both the engine and project are on drive F but the speed test was done on drive C (the default assumed location of the engine)
- In a system that two DDC are in same drive, the speed measured (if it was correct) would cause a race condition based on uncontrolled situations, such as if drive was busy when measuring, one DDC would become dominant causing the data to be copied to both of them on different times.
- It also says the order of lookup depends on speed of cache medium, overriding the user’s preference on how they ordered them in the list. In other word, a user can organize cache as
order Inner=EnginePak, Inner=CompressedPak, Inner=Pak, Inner=Localand there is no guaranty engine will look into EnginePak before CompressedPak and then Pak and then Local. With this in mind, and engine copying content from one cache to another, you will have at least 2 copies of cache (even 3 or 4) of same data which will use 2~4 times the size of the data in storage mediums. My 2 instances of DDC cache each has about 30GB and 35GB data in them which In normal conditions the one in engine folder should’ve stayed at 1.3GB at least.
What it should do instead
Measure the speed (fix it first) and only report it in the logs, so user know if there is a bottleneck and let them decide if they want or will be able to change the order of cache or find a faster storage.
It shouldn’t copy existing items from one cache to another at all. it should look into each cache in order of appearance in the list, regardless of their access speed, and if it found the entry it is looking for, stop looking further, and if not found otherwise, try to copy it to the first writable medium.
The current bugs:
- Stopping Zen will make the
InstalledDerivedDataBackendGraphto stop working and crashes the editor (error message is added at the end. I’m not exactly sure why, but it seems it could be either related to the random selection of the medium that it is trying to read first or theKeyLengthis broken. I’m leaning toward the brokenKeyLengthbecause other configurations which specify this are also not working. - I have tried many configuration combinations and even used the
Developer Assistantwith no success. It seems some combinations will not ever work at all. - In configurations there are multiple locations being used which makes configurations inconsistent. Some uses
../../../and there are instances of%ENGINEDIR%,%ENGINEVERSIONAGNOSTICUSERDIR%,%ENGINEVERSIONAGNOSTICINSTALLEDUSERDIR%or %GAMEDIR%. I couldn’t find the definition of either of these variables, in documentation but I hope they are what they supposed to be. - Some configurations have cache locations that are invalid. Since there are no proper document regarding this configuration, I cannot understand why
NoZenLocalFallbackconfiguration should have aZenSharedconfiguration in it. - In few cases after building project for hours, build was failed with no output due to not being able to write
compressed.ddcbecause it was locked by another process and upon checking that other process is the editor itself.
Lack of information in documentation
The documentation for data cache (link above) is not complete. It has no information on how to configure the cache if I just want to use a specific model and what are the options and their limitations, for example if I just want to use a compressed, what options are available, is it writable? if not, how to create one etc.
Just Zen auto start is disabled. No other chages in baseEngine.int:
Fatal error: [File:D:\build\++UE5\Sync\Engine\Source\Developer\DerivedDataCache\Private\DerivedDataBackends.cpp] [Line: 208]
Unable to use default cache graph 'InstalledDerivedDataBackendGraph' because there are no writable nodes available.Add -DDC-ForceMemoryCache to the command line to bypass this if you need access to the editor settings to fix the cache configuration.
Editor copying/caching engine content into project cache
