Nogbit
November 19, 2018, 5:04am
1
I’m working on a Paragon Hero Lab that is coming along nicely. However, now that I have all the heroes in the project the size of that directory is now 76GB. I was wondering if anyone has suggestions on optimizing this? The main bulk comes from each Hero’s Name/Characters/Global and Name/Characters/Herroes directories.
The Global directory for each character can be almost 1/3 of that characters total size. I was thinking that maybe these global things in the real Paragon project are truly global unlike what I have with them all downloaded and being unique to each hero directory.
Is there a way to collapse all these Hero directories?
FuTou
August 21, 2019, 9:25pm
2
Sorry to bring up an old post, but I think this is some important question…especially for indie devs.
Nogbit
October 24, 2019, 3:09am
3
No love I guess for this one as I revisit this project. Did you have any luck FuTou?
Nogbit
October 24, 2019, 6:32am
4
For anyone that stumbles upon this. I did a package build of a new empty c++ project that included only the Sparrow and Revenant characters.
The build/Content/Paks included one file, 5.37 GB.
If Epic is wondering why these assets arent used more this is why. It’s great to play with them in the editor and learn things (they are awesome), but the pak grows by 2GB or more per character.
Here is the console output of the build results.
Of that you can see that there is 4725mb of Texture Streaming and 1213mb of Texture Inline
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: Cook by the book total time in tick 3058.898629s total time 3065.734131
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: Peak Used virtual 9181MB Peak Used physical 8953MB
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: Hierarchy Timer Information:
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: Root: 0.000s (0)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: StartCookByTheBookTime: 6.550s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: GenerateAssetRegistryTime: 5.336s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: CleanSandboxTime: 0.045s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: ProcessingAccessedStrings: 0.005s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: CollectFilesToCook: 0.375s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: CookModificationDelegate: 0.002s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: GeneratePackageNames: 0.000s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: GenerateLongPackageName: 0.009s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: TickCookOnTheSide: 3058.511s (2213)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: LoadPackage: 172.130s (2212)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: CallBeginCacheForCookedPlatformData: 0.148s (2213)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: PostLoadPackageFixup: 0.001s (2213)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: SavingPackages: 2886.090s (1888)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: PrecachePlatformDataForNextPackage: 0.380s (3723)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: PrecachePlatformDataForNextNextPackage: 5.351s (3662)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: SaveCookedPackage: 2855.908s (3636)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: LoadPackage: 0.006s (3636)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: GEditorSavePackage: 2855.707s (3636)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: ConvertingBlueprints: 0.035s (3636)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: VerifyCanCookPackage: 0.020s (3623)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: ClearAllCachedCookedPlatformData: 0.143s (3636)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: EnqueueUnsavedPackages: 0.006s (34)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: GeneratingBlueprintAssets: 0.005s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: SavingCurrentIniSettings: 0.034s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: ProcessingAccessedStrings: 0.005s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: SavingAssetRegistry: 0.134s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: BuildChunkManifest: 0.025s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: SaveManifests: 0.005s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: SaveRealAssetRegistry: 0.088s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: WriteCookerOpenOrder: 0.016s (1)
UATHelper: Packaging (Windows (64-bit)): LogCook: Display: Done!
UATHelper: Packaging (Windows (64-bit)): LogSavePackage: Display: Took 0.103886s to verify the EDL loading graph.
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Misc Cook Stats
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: ===============
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: ShaderCompiler
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: BlockingTimeSec=2765.274692
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: AsyncCompileTimeSec=22823.230442
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: GlobalBeginCompileShaderTimeSec=25.376524
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: GlobalBeginCompileShaderCalls=54836
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: ProcessAsyncResultsTimeSec=0.263780
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: GlobalShader.Misc
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: ShadersCompiled=0
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: MeshMaterial.Misc
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: ShadersCompiled=54522
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: MaterialShader.Misc
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: ShadersCompiled=314
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Package.Load
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: NumPackagesLoaded=2338
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: LoadPackageTimeSec=172.560409
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Package.Save
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: NumPackagesSaved=3636
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: SavePackageTimeSec=2855.573198
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: TagPackageExportsPresaveTimeSec=2276.824890
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: TagPackageExportsTimeSec=2.026758
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: ResetLoadersForSaveTimeSec=0.036801
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: TagPackageExportsGetObjectsWithOuter=0.041838
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: TagPackageExportsGetObjectsWithMarks=0.004252
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: SerializeImportsTimeSec=0.000000
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: SortExportsSeekfreeInnerTimeSec=2.174803
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: SerializeExportsTimeSec=2.503110
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: SerializeBulkDataTimeSec=16.437074
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: AsyncWriteTimeSec=1.301596
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: MBWritten=5181.432996
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Package.DifferentPackagesSizeMBPerAsset
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Package.NumberOfDifferencesInPackagesPerAsset
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Package.PackageDifferencesSizeMBPerAsset
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Package.DiffTotal
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: NumberOfDifferentPackages=0
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: DifferentPackagesSizeMB=0.000000
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: NumberOfDifferencesInPackages=0
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: PackageDifferencesSizeMB=0.000000
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display:
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Cook Profile
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: ============
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0.CookWallTimeSec=3070.057001
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 0.StartupWallTimeSec=4.139037
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1.CookByTheBookTimeSec=3065.917948
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 0.StartCookByTheBookTimeSec=6.550206
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 0. 0.GameCookModificationDelegateTimeSec=0.002252
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 1.TickCookOnTheSideTimeSec=3059.082537
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 1. 0.TickCookOnTheSideLoadPackagesTimeSec=172.171864
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 1. 1.TickCookOnTheSideSaveCookedPackageTimeSec=2855.908859
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 1. 1. 0.TickCookOnTheSideResolveRedirectorsTimeSec=0.000000
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 1. 2.TickCookOnTheSideBeginPackageCacheForCookedPlatformDataTimeSec=5.863924
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 1. 3.TickCookOnTheSideFinishPackageCacheForCookedPlatformDataTimeSec=24.068116
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 2.TickLoopGCTimeSec=0.000000
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 3.TickLoopRecompileShaderRequestsTimeSec=0.022113
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 4.TickLoopShaderProcessAsyncResultsTimeSec=0.010475
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 5.TickLoopProcessDeferredCommandsTimeSec=0.000231
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: 0. 1. 6.TickLoopTickCommandletStatsTimeSec=0.000036
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display:
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: DDC Summary Stats
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: =================
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: TotalGetHits = 5011
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: TotalGets = 6896
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: TotalGetHitPct= 0.726653
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: LocalGetHitPct= 0.453161
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: SharedGetHitPct= 0.000000
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: OtherGetHitPct= 0.273492
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: GetMissPct = 0.273347
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: TotalPutHits = 2888
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: TotalPuts = 2888
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: TotalPutHitPct= 1.000000
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: PutMissPct = 0.000000
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display:
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: DDC Resource Stats
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: =======================================================================================================
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Asset Type Total Time (Sec) GameThread Time (Sec) Assets Built MB Processed
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: ---------------------------------- ---------------- --------------------- ------------ ------------
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Texture (Inline) 39.91 39.91 326 1213.91
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: SubUVAnimation 9.13 9.13 9 0.06
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: Texture (Streaming) 8.12 0.02 0 4725.80
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: MaterialShader 1.83 1.83 314 193.14
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: SkeletalMesh 1.04 1.04 2 126.22
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: StaticMesh 1.00 1.00 42 9.88
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: PhysX (BodySetup) 0.15 0.15 39 6.29
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: NavCollision 0.13 0.13 42 0.42
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: GlobalShader 0.02 0.02 0 3.42
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: SoundWave 0.01 0.01 645 18.50
UATHelper: Packaging (Windows (64-bit)): LogCookCommandlet: Display: DistanceField 0.00 0.00 0 0.00
Nogbit
November 26, 2020, 5:31pm
5
Updating this old post. I was able to automate the reduction in total size of the /Content directory from 72.8GB down to 49GB. This removed 6869 duplicate files. Enable the Python plugin, save this script to file on your drive. Then, in the Output Log, type the full path (with file name) to your script to run it (please read the comments before doing so).
https://gist.github.com/Nogbit/1c69b…
1 Like
Danestan
May 31, 2021, 3:46am
6
Thanks Nogbit for that one !
For others, who wanna try it :
As Nogbit said, read the script’s comments as there is a folder you may want to modify and some tweaking stuff.two plugins : Python Editor Script Plugin and Editor Scripting Utilities .Cmd , in your newly opened window, to select Python . Finally, you can type the full path to the script file you downloaded and wait for some time as Unreal does his thing.
Turkm4000
June 12, 2023, 5:02pm
7
Which one we should use there alot files!!!