Very slow derived data cache problem

Development Programming & Scripting C++
, , ,
1

Hi all!

I’ve been having some weird problems with Unreal Engine as of late and I don’t really know what’s going on, and I’d like to share my problem with the forums in case there’s someone out there who can shed light into this issues.

My main problem is that the current version of the Unreal Engine that I’m using is very slow when loading data, as in levels (for example), or when packaging the game. This hadn’t been an issue before, especially when I was using the binary version of the engine as provided through the Epic Games Launcher. Nowadays I’ve made the jump to the source code version, and it is this version which gives me problems. I’ll try to cite them below:

1/ When loading the editor after booting up the computer, as well as the Unreal Engine project, these take a long time. Longer than if I open the project and close it and then open it again, for instance. This seems to tell me that the derived data cache is either not being accessed and the engine has to recreate those assets, or that the access to the derived data cache folder is very slow and that’s why it takes so long.

However, I’m more inclined to think that the derived data cache isn’t being accessed as if I close and reload the project those assets take less time to load - probably because they are still in memory, which could point to them being recreated? Not sure about this.

2/ When packaging the project I get the following message all over the place:

Warning: D:/Unreal/PlanCity/PlanCity/LocalDerivedDataCache is very slow (0.02MB/s) when accessing D:/Unreal/PlanCity/PlanCity/LocalDerivedDataCache/8/8/4/TEXTURE_68A083899C6F4316B8CE0E2958EDE2C2_BC5_567955614D2260985028FA89B708C677___ABDDB91D6707A292EB0D9F7B5B9E29AAD6222E74.udd, consider disabling it.

So I’m not sure why this is happening, and I’m sure it must be related to my first problem. It seems like that folder is very slow to access, which is weird, as I have other projects using a binary version of the engine stored in the D drive which don’t show that problem. With all of this in mind, I guess the problem really lies in the engine and how it’s accessing the derived data cache folder? The only major difference between the source code engine and the binary ones is that the latter one has been compiled in the D drive, a mechanical hard drive, and the others live on the SSD. However, all of the derived data cache folder for every project that I create gets stored in the D drive, so not sure how that would be a problem if at all.

What do you guys think? Anyone encountered something similar before? What should I do?! :S

Thanks for all of your time in advance by the way :slight_smile:

2

By default, the launcher builds store the DDC for all projects in a shared folder in the AppData folder, which is on drive C. Check it out and you’ll see it’s probably quite large. Your D is probably too slow or is defective, because I’ve never seen that message even when forcing the DDC to a mechanical drive.

In the editor settings you can set up a custom location for the shared DDC and move it to your SSD.

3

Hey Manoel!

Never got around to saying thanks for the reply, so here’s that thanks - better late than never :stuck_out_tongue:

In any case, the problem ended up being a bit weirder than expected. I found the solution thanks to Epic, and you can check their replies in here: https://udn.unrealengine.com/questions/517221/view.html

Thanks for your input though, really appreciate it :slight_smile:

4

Hello Brais, could you briefly outline the steps you took to resolve the issue, the linked website is currently not available for my account.

1 Like
5

Perhaps you missed my reply, would be really cool if you could point out how you addressed the issue of slow derived data cache files, thanks.

1 Like
6

FYI @unit23 in a nutshell the solution suggested on UDN was to move the shared DDC to an SSD, possibly Manoel was correct in his guess that the HDD was defective.

7

Thanks, using already a SSD for the cache, but the Engine runs on a hard drive. No errors found by Windows. I will move the Engine also to the SSD I guess…

8

Moving Engine and Project to the same partition almost resolves all of the derived cache warnings during cooking, four warnings remained.

9

Today I had a server memory leak, which subsequently corrupted a blueprint structure. Still investigating…

10

Hi all!

Just seen these messages today, let me try to answer your questions @unit23! And thanks @rmclachlan_TCR for jumping in. The problem was that my system was disk bound - lots of parallel access to data that was scattered throughout the disk really slowed it down. This isn’t an issue that showed when using binary versions of the engine, only on source code builds. I couldn’t find an explanation as to why that was the case, and I decided to just buy another SSD to get rid of the problem. Mind you, I don’t think you can get rid of that problem, but it would be interesting to know why that happens in a source code build and not on a binary version.

Luckily the HDD wasn’t defective though, so it can still be used for other stuff… just not Unreal projects! Which makes sense anyways, as it was a HDD designed for storage more than anything else. Really strange though how its behaviour changed depending on which engine version you were using.

Best of luck with your situation! After reading your posts, I would just suggest to go for an SSD. Makes a big difference performance-wise.

Best,

Brais

11

So it turns out that a new SSD was in fact corrupt, subsequently causing the very slow derived data cache situation.