Announcement

Collapse
No announcement yet.

Reducing the file size of a Paragon hero assets in a single project?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Reducing the file size of a Paragon hero assets in a single project?

    I'm working on a Paragon Hero Lab that is coming along nicely. However, now that I have all the heroes in the project the size of that directory is now 76GB. I was wondering if anyone has suggestions on optimizing this? The main bulk comes from each Hero's Name/Characters/Global and Name/Characters/Herroes directories.

    The Global directory for each character can be almost 1/3 of that characters total size. I was thinking that maybe these global things in the real Paragon project are truly global unlike what I have with them all downloaded and being unique to each hero directory.

    Is there a way to collapse all these Hero directories?
    What advice do you have if any?

    #2
    Sorry to bring up an old post, but I think this is some important question...especially for indie devs.

    Comment


      #3
      No love I guess for this one as I revisit this project. Did you have any luck FuTou?

      Comment


        #4
        For anyone that stumbles upon this. I did a package build of a new empty c++ project that included only the Sparrow and Revenant characters.

        The build/Content/Paks included one file, 5.37 GB.

        If Epic is wondering why these assets arent used more this is why. It's great to play with them in the editor and learn things (they are awesome), but the pak grows by 2GB or more per character.





        Here is the console output of the build results.

        Of that you can see that there is 4725mb of Texture Streaming and 1213mb of Texture Inline

        Code:
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display: Cook by the book total time in tick 3058.898629s total time 3065.734131
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display: Peak Used virtual 9181MB Peak Used physical 8953MB
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display: Hierarchy Timer Information:
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:   Root: 0.000s (0)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:     StartCookByTheBookTime: 6.550s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       GenerateAssetRegistryTime: 5.336s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       CleanSandboxTime: 0.045s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:         ProcessingAccessedStrings: 0.005s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       CollectFilesToCook: 0.375s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:         CookModificationDelegate: 0.002s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:         GeneratePackageNames: 0.000s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       GenerateLongPackageName: 0.009s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:     TickCookOnTheSide: 3058.511s (2213)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       LoadPackage: 172.130s (2212)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       CallBeginCacheForCookedPlatformData: 0.148s (2213)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       PostLoadPackageFixup: 0.001s (2213)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       SavingPackages: 2886.090s (1888)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:         PrecachePlatformDataForNextPackage: 0.380s (3723)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:         PrecachePlatformDataForNextNextPackage: 5.351s (3662)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:         SaveCookedPackage: 2855.908s (3636)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:           LoadPackage: 0.006s (3636)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:           GEditorSavePackage: 2855.707s (3636)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:             ConvertingBlueprints: 0.035s (3636)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:             VerifyCanCookPackage: 0.020s (3623)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:         ClearAllCachedCookedPlatformData: 0.143s (3636)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:         EnqueueUnsavedPackages: 0.006s (34)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:     GeneratingBlueprintAssets: 0.005s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:     SavingCurrentIniSettings: 0.034s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       ProcessingAccessedStrings: 0.005s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:     SavingAssetRegistry: 0.134s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       BuildChunkManifest: 0.025s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       SaveManifests: 0.005s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       SaveRealAssetRegistry: 0.088s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display:       WriteCookerOpenOrder: 0.016s (1)
        UATHelper: Packaging (Windows (64-bit)):   LogCook: Display: Done!
        UATHelper: Packaging (Windows (64-bit)):   LogSavePackage: Display: Took 0.103886s to verify the EDL loading graph.
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Misc Cook Stats
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: ===============
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: ShaderCompiler
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     BlockingTimeSec=2765.274692
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     AsyncCompileTimeSec=22823.230442
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     GlobalBeginCompileShaderTimeSec=25.376524
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     GlobalBeginCompileShaderCalls=54836
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     ProcessAsyncResultsTimeSec=0.263780
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: GlobalShader.Misc
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     ShadersCompiled=0
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: MeshMaterial.Misc
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     ShadersCompiled=54522
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: MaterialShader.Misc
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     ShadersCompiled=314
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Package.Load
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     NumPackagesLoaded=2338
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     LoadPackageTimeSec=172.560409
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Package.Save
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     NumPackagesSaved=3636
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     SavePackageTimeSec=2855.573198
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     TagPackageExportsPresaveTimeSec=2276.824890
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     TagPackageExportsTimeSec=2.026758
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     ResetLoadersForSaveTimeSec=0.036801
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     TagPackageExportsGetObjectsWithOuter=0.041838
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     TagPackageExportsGetObjectsWithMarks=0.004252
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     SerializeImportsTimeSec=0.000000
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     SortExportsSeekfreeInnerTimeSec=2.174803
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     SerializeExportsTimeSec=2.503110
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     SerializeBulkDataTimeSec=16.437074
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     AsyncWriteTimeSec=1.301596
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     MBWritten=5181.432996
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Package.DifferentPackagesSizeMBPerAsset
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Package.NumberOfDifferencesInPackagesPerAsset
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Package.PackageDifferencesSizeMBPerAsset
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Package.DiffTotal
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     NumberOfDifferentPackages=0
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     DifferentPackagesSizeMB=0.000000
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     NumberOfDifferencesInPackages=0
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:     PackageDifferencesSizeMB=0.000000
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Cook Profile
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: ============
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0.CookWallTimeSec=3070.057001
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 0.StartupWallTimeSec=4.139037
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1.CookByTheBookTimeSec=3065.917948
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 0.StartCookByTheBookTimeSec=6.550206
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 0. 0.GameCookModificationDelegateTimeSec=0.002252
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 1.TickCookOnTheSideTimeSec=3059.082537
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 1. 0.TickCookOnTheSideLoadPackagesTimeSec=172.171864
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 1. 1.TickCookOnTheSideSaveCookedPackageTimeSec=2855.908859
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 1. 1. 0.TickCookOnTheSideResolveRedirectorsTimeSec=0.000000
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 1. 2.TickCookOnTheSideBeginPackageCacheForCookedPlatformDataTimeSec=5.863924
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 1. 3.TickCookOnTheSideFinishPackageCacheForCookedPlatformDataTimeSec=24.068116
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 2.TickLoopGCTimeSec=0.000000
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 3.TickLoopRecompileShaderRequestsTimeSec=0.022113
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 4.TickLoopShaderProcessAsyncResultsTimeSec=0.010475
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 5.TickLoopProcessDeferredCommandsTimeSec=0.000231
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:  0. 1. 6.TickLoopTickCommandletStatsTimeSec=0.000036
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: DDC Summary Stats
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: =================
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: TotalGetHits  =      5011
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: TotalGets     =      6896
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: TotalGetHitPct=  0.726653
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: LocalGetHitPct=  0.453161
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: SharedGetHitPct=  0.000000
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: OtherGetHitPct=  0.273492
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: GetMissPct    =  0.273347
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: TotalPutHits  =      2888
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: TotalPuts     =      2888
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: TotalPutHitPct=  1.000000
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: PutMissPct    =  0.000000
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display:
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: DDC Resource Stats
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: =======================================================================================================
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Asset Type                          Total Time (Sec)  GameThread Time (Sec)  Assets Built  MB Processed
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: ----------------------------------  ----------------  ---------------------  ------------  ------------
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Texture (Inline)                               39.91                  39.91           326       1213.91
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: SubUVAnimation                                  9.13                   9.13             9          0.06
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: Texture (Streaming)                             8.12                   0.02             0       4725.80
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: MaterialShader                                  1.83                   1.83           314        193.14
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: SkeletalMesh                                    1.04                   1.04             2        126.22
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: StaticMesh                                      1.00                   1.00            42          9.88
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: PhysX (BodySetup)                               0.15                   0.15            39          6.29
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: NavCollision                                    0.13                   0.13            42          0.42
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: GlobalShader                                    0.02                   0.02             0          3.42
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: SoundWave                                       0.01                   0.01           645         18.50
        UATHelper: Packaging (Windows (64-bit)):   LogCookCommandlet: Display: DistanceField                                   0.00                   0.00             0          0.00

        Comment


          #5
          Updating this old post. I was able to automate the reduction in total size of the /Content directory from 72.8GB down to 49GB. This removed 6869 duplicate files. Enable the Python plugin, save this script to file on your drive. Then, in the Output Log, type the full path (with file name) to your script to run it (please read the comments before doing so).

          https://gist.github.com/Nogbit/1c69b...e8cfe0f9edea60

          Comment

          Working...
          X