Announcement

Collapse
No announcement yet.

Clean me up, Scotty. Project cleaning and organising Tool

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    [WIP] Clean me up, Scotty. Project cleaning and organising Tool

    Hi there!

    I found the awesome Editor Scripting possibilities which came in 4.20. It startet as a small script to help me organising my own files. I quickly realized the endless possibilities of it. Here are some videos I created to show the curernt state of the plugin.

    Clean Up and organise all assets
    Remove Duplicates
    Delete empty folders, set sRGB values and add prefixes to all assets
    Remove unused assets

    Right now it is possible to:
    • Batch rename all your files directly in unreal.
    • Find duplicates in your project, relink everythin to only just one single file and delete all the duplicates. (Defined by same name. This is really usefull if you are working with a lot of asset packs
    • Automatically move all your Assets in appropriate folders. Creates a folder structure and moves alll BP`s in the Blueprint folder etc.
    • Define a pattern (like "OcclusionRoughnessMetallic", or "ORM" or "_Mask" ) and turn off sRGB on all those textures with one click
    • Delete all empty folders in the project
    • Add Prefixes to all assets according to the official Unreal naming convention guide (Or the one you specify)
    • Combine different assets to just one asset and replace their dependencies. For Example remove all instances of a glass shader and only use one of it in your project.
    • Cleanup all unused assets. This script is the biggest part. It checks all soft and hard references. So for example, if an actor is not in your current level, but you spawn it from another blueprint at runtime it is not sent to trash. Even if it is in another level. You can use this opreation on a single folder or the whole project folder. You can also lock specific folders.
    • Find textures that are not a power of 2

    Next step I have on my list are:
    • Would love to set folder color. But this is currently not yet possible at runtime from code.

    I tried to contact unreal developers to get some information about the asset regestry. But I was just forwarded to this forum with a standart reply. So I guess I´ll need to figure this one out by myself.

    Here is the documentation: Documentation

    Added Support on Discord for faster Feedback
    Last edited by Andyhdr; 08-17-2019, 02:19 AM.

    #2
    Hi Andy thanks for this will certainly check it out. As a suggestion, another thing are materials (and their textures), which are not easily identifiable but are essentially redundant. These include the base mannequin his bp and asset relations, glass materials, some basic materials for basic coloring, which are part of many asset packs. However, usually it is enough to use a few glass materials, only one or two mannequins, and colors can also be shared. So what would be nice to make a list of the most common materials, textures, bp, to produce an overview list of them, and then you could decide which to delete and what "double" you would take as replacement file.

    So this was just an idea I got when I read yesterday your comment in the other thread about deleting doubles. Not sure how feasible this is, but it is certainly something where you can reduce project file size, and make the project more manageable.
    LEGENDS OF EPICA | CS - SPACE ALPHA

    Comment


      #3
      Awesome. That´s a great idea. Thank you for bringing it up! I will have a look into it once this "delete unused nodes" monster is done.
      For today I added a function to delete all empty folders in the project. That´s especially handy after moving the files with my cleanup function.

      Comment


        #4
        would love to have a tool like that !
        www.ofpawnsandkings.de

        Comment


          #5
          Andyhdr I would put a exclusion list for deletion, because some assets are referenced at run time in C++ code.
          Nilson Lima
          Technical Director @ Rigel Studios Ltda - twitter: @RigelStudios
          Join us at Discord: https://discord.gg/uFFSEXY

          UE4 Marketplace: Cloudscape Seasons
          supporting: Community FREE Ocean plugin

          Comment


            #6
            Originally posted by NilsonLima View Post
            Andyhdr I would put a exclusion list for deletion, because some assets are referenced at run time in C++ code.
            You are right. It happened to me yesterday. Needed to reinstall the engine today because of that. So next thing is an exclusion list of directories

            Comment


              #7
              So I manged to write a script that deletes all unused assets in the project or in a specified directory. (Actually I´m creating a trash folder and move everything in there so you can have a second check whether you really want to delete it). It works perfectly... But sadly only if you have one level. It is not able to crosscheck between all levels in your project. A solution could be to open every Level, make a List out off all assets used there and then run the script...Do you think it is worth digging into this approach?

              Comment


                #8
                Sounds very useful - will keep an eye on it!
                [WIP] Procedural City Generator | RPG AI
                [MARKETPLACE] Animal Behavior Kit | Space Shooter Template | Procedural Foliage Tool | Procedural Park
                [FREE] Modular Road Tool | Action Platformer Template | Radar BP | Free Birds
                Join our Discord

                Comment


                  #9
                  Originally posted by Andyhdr View Post
                  So I manged to write a script that deletes all unused assets in the project or in a specified directory. (Actually I´m creating a trash folder and move everything in there so you can have a second check whether you really want to delete it). It works perfectly... But sadly only if you have one level. It is not able to crosscheck between all levels in your project. A solution could be to open every Level, make a List out off all assets used there and then run the script...Do you think it is worth digging into this approach?
                  I believe you need to check the same places where levels are listed/registered for packaging in the project settings to find out which ones you need to consider. This includes the assigned editor/client/server start levels, everything listed in the package maps list and then stuff found by the primary asset manager for always packaging.

                  Comment


                    #10
                    Originally posted by Manoel.Neto View Post

                    I believe you need to check the same places where levels are listed/registered for packaging in the project settings to find out which ones you need to consider. This includes the assigned editor/client/server start levels, everything listed in the package maps list and then stuff found by the primary asset manager for always packaging.
                    I agree, this would be the most desired. It can also handle a list of levels to check maybe?
                    Nilson Lima
                    Technical Director @ Rigel Studios Ltda - twitter: @RigelStudios
                    Join us at Discord: https://discord.gg/uFFSEXY

                    UE4 Marketplace: Cloudscape Seasons
                    supporting: Community FREE Ocean plugin

                    Comment


                      #11
                      Originally posted by Manoel.Neto View Post

                      I believe you need to check the same places where levels are listed/registered for packaging in the project settings to find out which ones you need to consider. This includes the assigned editor/client/server start levels, everything listed in the package maps list and then stuff found by the primary asset manager for always packaging.
                      Thanks for your feedback guys. It´s much aprreciated. Actually I forgot about the "always package" assets. That´s a great hint, thank you.
                      I would also love to use an array of levels. But as for now the function I´m using don´t support any input. It just checks the currently opened level. But I´m sure I´ll find a workaround for that. I`ll kepp you updated

                      Comment


                        #12
                        What does this do exactly:
                        • Find duplicates in your project, relink everythin to only just one single file and delete all the duplicates.


                        for e.g. I have a texture in the starter pack and the same identical texture (just different file location) in an asset-pack.
                        what criteria makes it a "duplicate" and which file is deleted, and what references are all changed (e.g. in C++ code, unopened maps) and how long does this take compared to the (slow) mechanics already implemented in UEd (redirectors)?

                        Comment


                          #13

                          for e.g. I have a texture in the starter pack and the same identical texture (just different file location) in an asset-pack.
                          what criteria makes it a "duplicate" and which file is deleted, and what references are all changed (e.g. in C++ code, unopened maps) and how long does this take compared to the (slow) mechanics already implemented in UEd (redirectors)?
                          Hi Pinotti, the scripts automatically searches for all duplicates in your project. That´s the benefit compared to manually searching, deleting and relinking all files. A duplicate is defined by the same asset name like the textures from the starter content also used by other asset packs. All the references inside of Unreal are changed and relinked. Even the ones used in unloaded levels.
                          In C++ code, if the asset was hardcoded, it is not affected. I´m only looking into BP Projects right now.

                          Comment


                            #14
                            Originally posted by Andyhdr View Post

                            Thanks for your feedback guys. It´s much aprreciated. Actually I forgot about the "always package" assets. That´s a great hint, thank you.
                            I would also love to use an array of levels. But as for now the function I´m using don´t support any input. It just checks the currently opened level. But I´m sure I´ll find a workaround for that. I`ll kepp you updated
                            IMO you don't really need to go that far. Just make sure to not delete anything that would otherwise get packaged as per the project's settings (since Unreal won't package any maps other than the start maps unless told otherwise). All that stuff can be accessed via C++.

                            Comment


                              #15
                              So, big news. I took all the testers feedback and essentially reworked the system. It´s much faster and stable now. Getting closer to be production ready. What I did in this update:
                              • Reworked the system to be more efficent and safe
                              • Added the much needed "Delete unused assets" feature (this time with a complete check for all soft and hard references). All unused assets (except the ones in locked folders) will be moved to a newly created trash folder. I will create a new video today showcasing this feature.
                              • Gave the tool to another 15 testers for feedback. Thank you for being part in the development!
                              Right now it seems like it´s not possible to add dialog boxes with editor scripting. But I found some promising changelogs for 4.22. I will add them once they are available.

                              Comment

                              Working...
                              X