Shared DDC is not being cleaned up

Development Pipeline & Plugins
, ,
1

Hi!

As far as I understand, shared DDC is supposed to be deleting old unused files as a part of maintenance for every cook invocation that is referencing that DDC location.

Yet our shared DDC has files older than a year, and I don’t see log messages like the following

`Maintenance finished in A and deleted B files with total size C MiB and D empty folders`

Our Base.Engine.ini and project’s DefaultEngine.ini looks like this:

BaseEngine.ini
[DerivedDataBackendGraph]
Root=(Type=Hierarchical, Inner=Pak, Inner=EnginePak, Inner=Local, Inner=Shared, Inner=Cloud)
Local=(Type=FileSystem, UnusedFileAge=34, PromptIfMissing=true, Path=%ENGINEDIR%DerivedDataCache, EnvPathOverride=UE-LocalDataCachePath, EditorOverrideSetting=LocalDerivedDataCache, CommandLineOverride=LocalDataCachePath)
Shared=(Type=FileSystem, UnusedFileAge=10, FoldersToClean=10, ConsiderSlowAt=70, Path=?EpicDDC, EnvPathOverride=UE-SharedDataCachePath, EditorOverrideSetting=SharedDerivedDataCache, CommandLineOverride=SharedDataCachePath)
Cloud=(Type=Cloud, ServerID=Default, EnvHostOverride=UE-CloudDataCacheHost, CommandLineHostOverride=CloudDataCacheHost, CommandLineOAuthSecretOverride="CloudDataCacheOAuthSecret", OAuthAccessTokenEnvOverride="UE-CloudDataCacheAccessToken", EnvHttpVersionOverride="UE-CloudDataCacheHttpVersion", CommandLineHttpVersionOverride="CloudDataCacheHttpVersion")
Pak=(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
EnginePak=(Type=ReadPak, Filename="%ENGINEDIR%DerivedDataCache/DDC.ddp")

DefaultEngine.ini
[DerivedDataBackendGraph]
Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=10, MaxFileChecksPerSec=1, ConsiderSlowAt=70, PromptIfMissing=false, Path="\\[netshare]\Cache\DerivedDataCache\5.3", EnvPathOverride=UE-SharedDataCachePath, EditorOverrideSetting=SharedDerivedDataCache)

Is there anything that could be preventing unused files from being deleted?

Thank you,

Dmitry

2

We have noticed that simply having Unreal Editor open produces the following messages in the Output Log:

LogDerivedDataCache: //[netshare]/Cache/DerivedDataCache/5.3: Maintenance finished in +00:16:50.575 and deleted 10 files with total size 4 MiB and 0 empty folders. Scanned 810 files in 1223 folders with total size 202 MiB.

Which is good! But seems a bit low though. Should we bump `FoldersToClean` or `MaxFileChecksPerSec` to get more files removed?

Edit: “files older than a year” - these are actually all TestData files, guess they are created once and never deleted?

3

Hi,

the default settings are intended for largers team, with the goal that each Editor process does incremental and throttled background cleanups periodically. It is expected that each cleanup would only do a small amount of work.

The idea is that with all team members doing this the DDC should stay up-to-date without requiring dedicated maintenance that might affect the editor performance for any of the team members. If this does not happen for you, you might want to increase the amount of files scanned.

You can also set up a dedicated cleaning run by setting the “Clean” option to true (which will force a blocking maintenance at startup) and increasing the FoldersToClean and MaxFileChecksPerSec values to much larger values.A commandlet run with those settings can be used to trigger a maintenance from a CI machine.

Starting from 5.5. we also recommend everyone to switch to a Zen Storage Server as DDC, since it significantly improves performance, allows automatic maintenance on the server and supports many of our more recent features and cooking and DDC.

Kind Regards,

Sebastian

4

> Just to double-check, from 5.5 Zen can be used both locally and as a Shared DDC, correct?

Yes, that’s correct.

Technically, it will likely work in 5.3, but we consider the local use case production-ready in 5.4 and the shared one production-ready in 5.5.

5

Hi Sebastian,

Thank you for the detailed answer!

We’ll try running CI maintenance with more folders & files scanned, perhaps with a smaller MaxFileAgeInDays as well.

Zen Storage Server is on our roadmap, as soon as we can upgrade from 5.3.

Just to double-check, from 5.5 Zen can be used both locally and as a Shared DDC, correct?

Thanks again,

Dmitry

6

(post deleted by author)