Announcement

Collapse
No announcement yet.

[Plugin] ZipUtility (7zip)

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • [Plugin] ZipUtility (7zip)

    Hey UE developers! A certain vr project of mine needed the ability to peek into archives and extract them efficiently and I couldn't find any good zip solution for UE4 so I decided to port 7zip, update it for VS2015 and make it easy to use. I hope this will be useful to some of you . If you can improve it, please make pull requests!


    An event-driven, multi-threaded, C++ & blueprint accessible 7zip archiver and file manipulation plugin for Unreal Engine 4. Built on 7zip-cpp modernization of the SevenZip++.

    Supports the following compression algorithms: Zip, 7-Zip, GZip, BZip2, RAR(decompress only), TAR, ISO, CAB, LZMA, LZMA86.

    Plugin works in Windows only.

    Download
    Stable


    Unstable
    Master Branch

    Documentation and Resources
    Github Repository

    Quick Install & Setup
    1. Download
    2. Create new or choose project.
    3. Browse to your project folder (typically found at Documents/Unreal Project/{Your Project Root})
    4. Copy Plugins folder into your Project root.
    5. Restart the Editor and open your project again. Plugin is now ready to use.


    Basic Usage - Blueprint
    After the quick setup, you will have new functions available to you in blueprint



    if you wish to receive callback events add ZipUtilityInterface to your blueprint class and pass a reference to self into your function calls



    which exposes these events



    so to e.g. Unzip a file you simply do



    Note that the event callbacks are optional and are called asynchronously on your game thread allowing you to easily show a progress indicator. The plugin can handle Zipping, Unzipping, Listing contents, Making Folders and Moving/Renaming Files.

    For more documentation, read the github repo readme, the main source of documentation.

    Changelist

    0.3.1
    -Fixes for packaging

    0.3.0
    -Compile fixes for 4.15
    -Added scaffold for status return on ZipUtility actions
    -misc internal changes from 0.2.2 to 0.2.6

    0.2.2
    -Thanks to #4 (@hyperdr1ve) the plugin now supports multiple simultaneous operations working independently on their own threads with proper callbacks.
    -Added windows listing folder content function with FileListInterface expected to receive the content data

    0.2.0
    -Windows API functions split into WindowsFileUtility module
    -Added a folder watcher interface (IFolderWatchInterface) and bp library function which will allow you to watch a folder for file changes and respond to such events e.g. directory changed or file changed
    -ZipUtility trimmed to only zipping/archive functions
    -Thanks to #1 (@deams51) now supports extracting a single file
    -Unreal engine 4.14 compile fixes

    0.1.1
    -Added delete file/folder functions
    -Added UnzipTo
    -Merged UnzipWithFormat with Unzip. Compression set to unknown will auto-detect compression.
    -All callbacks now return with the archive path, allowing you to distinguish between simultaneous actions.
    -Fixed callback consistency, unzip and zip will emit all progress updates and list callback will return with OnDone when listing has completed.
    -Update to 7z-cpp bind to v0.2.0
    -Added macro definitions for C++ integration

    0.1.0
    -First public commit for UE4.10
    Last edited by getnamo; 05-09-2017, 02:47 PM.
    Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

  • #2
    VR comic reader? :P


    Awesome work man
    Storyteller - An immersive VR audiobook player

    Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

    Comment


    • #3
      This is really, neat! I have been thinking about dealing with zipped files and unreal but decided to not bother (as it never got to the point of needing it). Awesome utility, now ill have to check it out!
      Twitter - @TheKRushin

      Feel free to ask questions and come learn to be an #UnrealDev

      How to report a Bug

      Comment


      • #4
        Originally posted by KRushin View Post
        This is really, neat! I have been thinking about dealing with zipped files and unreal but decided to not bother (as it never got to the point of needing it). Awesome utility, now ill have to check it out!
        Let me know if you think anything is missing, feel free to modify/make pull requests.

        Originally posted by n00854180t View Post
        VR comic reader? :P


        Awesome work man
        Cheers! Not quite comics, although that sounds pretty cool. It's more to do with automatic download and checking if they're VR experiences and moving them into place/etc. If you're interested in the use case, keep an eye out in the Nexus thread.
        Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

        Comment


        • #5
          Are there any license concerns here? I thought rar and/or 7zip has license issues

          Comment


          • #6
            Originally posted by getnamo View Post
            Let me know if you think anything is missing, feel free to modify/make pull requests.



            Cheers! Not quite comics, although that sounds pretty cool. It's more to do with automatic download and checking if they're VR experiences and moving them into place/etc. If you're interested in the use case, keep an eye out in the Nexus thread.
            I'd love to see Nexus support Hydra and other motion controls in addition to Leap Motion.
            Storyteller - An immersive VR audiobook player

            Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

            Comment


            • #7
              Originally posted by n00854180t View Post
              I'd love to see Nexus support Hydra and other motion controls in addition to Leap Motion.
              That is in the works, multiplayer and easy portal downloads first though



              Originally posted by John Vanderbeck View Post
              Are there any license concerns here? I thought rar and/or 7zip has license issues
              The plugin modifications and 7zip-cpp are MIT, SevenZip++ is public domain.

              7-Zip is LGPL + UnRar restriction which simply means you can't compress RAR or make a RAR archiver (this plugin won't allow you to compress RAR anyway, just decompress), decompressing/unarchiving is allowed and you just need to retain that specific notice in your license

              The unRAR sources cannot be used to re-create the RAR compression algorithm,
              which is proprietary. Distribution of modified unRAR sources in separate form
              or as a part of other software is permitted, provided that it is clearly
              stated in the documentation and source comments that the code may
              not be used to develop a RAR (WinRAR) compatible archiver.


              You can comment one line of code to remove the Unrar restriction using 7za.dll instead of 7z.dll by changing line 138 and 139 in ZipFileFunctionLibrary, this will however limit your archive support to .7z only due to how 7za.dll is compiled (you can in theory re-enable everything but RAR, but you will need to recompile a compatible 7zip dll yourself using the LZMA sdk which is public domain), for my own purposes the unrar restriction is acceptable and I prefer the wider archive support so I default to 7z.dll.

              Regarding LGPL, because this plugin is using the com-like port interface to the 7zip .dll it doesn't cause the rest of the code (e.g. modifications and any unrelated code) to fall under LGPL. From the wiki on LGPL: a standalone executable that dynamically links to a library through a .so, .dll, or similar medium is generally accepted as not being a derivative work as defined by the LGPL. It would fall under the definition of a "work that uses the Library".

              Paragraph 5 of the LGPL version 2.1 states: A program that contains no derivative of any portion of the Library, but is designed to work with the Library by being compiled or linked with it, is called a "work that uses the Library". Such a work, in isolation, is not a derivative work of the Library, and therefore falls outside the scope of this License.

              While not relevant, that dll barrier is actually another static library (7zip-cpp) away from the plugin in release mode and another dynamic link barrier in editor mode (this plugin .dll).

              TL;DR:
              It may be not trivial to parse but you are ok with using this plugin commercially with the license file included, that said I'm not a lawyer and you should make your own judgement, refer to the plugin license file for details.
              Last edited by getnamo; 01-06-2016, 06:26 AM.
              Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

              Comment


              • #8
                Update to 0.1.1
                -Added delete file/folder functions
                -Added UnzipTo
                -Merged UnzipWithFormat with Unzip. Compression set to unknown will auto-detect compression.
                -All callbacks now return with the archive path, allowing you to distinguish between simultaneous actions.
                -Fixed callback consistency, unzip and zip will emit all progress updates and list callback will return with OnDone when listing has completed.
                -Update to 7z-cpp bind to v0.2.0
                -Added macro definitions for C++ integration


                This update brings fixes from 7z-cpp and improves the reliability of callbacks. You can now also call simultaneous actions and distinguish between results and progress updates via the archive path string parameter attached to each callback.

                New file manipulation functions (delete file/folder) allow you to clean up temporary files e.g. an archive after you have unzipped. UnzipTo function allows you to unzip to a different location than where the archive is present.

                Note that DeleteFolderRecursively can be dangerous, but it shouldn't allow you to delete outside your game directory (basic sandboxing).

                New Callbacks Example


                New UnzipTo Function Example


                Also changed git structure to better follow distinction between source and releases, testing the waters to see if this is more easily maintainable than what I was doing earlier.
                Last edited by getnamo; 01-17-2016, 09:11 PM.
                Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                Comment


                • #9
                  Hi getnamo! I really like your Plugin and have been using it c++ side. I'm now trying to package my game and am getting this error

                  Code:
                  UE4-ZipUtility.lib(Module.ZipUtility.cpp.obj) : error LNK2001: unresolved external symbol "class ATL::CAtlBaseModule ATL::_AtlBaseModule" (?_AtlBaseModule@ATL@@3VCAtlBaseModule@1@A)
                  7zpp_u.lib(stdafx.obj) : error LNK2001: unresolved external symbol "class ATL::CAtlBaseModule ATL::_AtlBaseModule" (?_AtlBaseModule@ATL@@3VCAtlBaseModule@1@A)
                  Any clues as to what is going on? I've tried all changing all kinds of plugin/Engine settings but can't get it to work. I see that CAtlBaseModule is a Windows thing, but not sure what to do with that information.

                  Thanks!

                  Comment


                  • #10
                    Originally posted by lukefwilson View Post
                    Hi getnamo! I really like your Plugin and have been using it c++ side. I'm now trying to package my game and am getting this error

                    Code:
                    UE4-ZipUtility.lib(Module.ZipUtility.cpp.obj) : error LNK2001: unresolved external symbol "class ATL::CAtlBaseModule ATL::_AtlBaseModule" (?_AtlBaseModule@ATL@@3VCAtlBaseModule@1@A)
                    7zpp_u.lib(stdafx.obj) : error LNK2001: unresolved external symbol "class ATL::CAtlBaseModule ATL::_AtlBaseModule" (?_AtlBaseModule@ATL@@3VCAtlBaseModule@1@A)
                    Any clues as to what is going on? I've tried all changing all kinds of plugin/Engine settings but can't get it to work. I see that CAtlBaseModule is a Windows thing, but not sure what to do with that information.

                    Thanks!
                    Seems to be that we're missing links to atls.lib (https://blogs.msdn.microsoft.com/mik...winmoduleinit/) or some form of basic windows lib/dependency. The plugin uses some windows specific functions to move files around (convenience functions found at https://github.com/getnamo/ZipUtilit...brary.cpp#L398), if you're not using them you could disable them or consider maybe finding a good pull request to make it less windows specific?

                    Otherwise the lib dependency needs to be resolved.

                    Haven't tried to package this plugin before, so you're first to try this out
                    Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                    Comment


                    • #11
                      Great. Looks like we did the same research on this one. Thanks for pointing out where the windows functions are being called. I'm changing my strategy and am not using the plugin anymore (for a host of other reasons not related to the plugin itself), but it looks like a solvable problem.

                      Comment


                      • #12
                        @Getnamo - I get a crash if I hit play in editor, do some unzipping work, stop, then hit play again. Seems like the thread isn't cleaned up always when ending the in-editor play.

                        Thanks for this btw, it works great for my comic book reader.
                        Last edited by n00854180t; 08-21-2016, 02:52 PM.
                        Storyteller - An immersive VR audiobook player

                        Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

                        Comment


                        • #13
                          Originally posted by n00854180t View Post
                          @Getnamo - I get a crash if I hit play in editor, do some unzipping work, stop, then hit play again. Seems like the thread isn't cleaned up always when ending the in-editor play.

                          Thanks for this btw, it works great for my comic book reader.
                          Got any logs for when these crashes happen?
                          Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                          Comment


                          • #14
                            Code:
                            2>A:\Work\Projects\NY_Loft - PC\Plugins\ZipUtility\Source\ZipUtility\Private\ZipFileFunctionLibrary.cpp(4): fatal error C1083: Cannot open include file: 'ListCallback.h': No such file or directory
                            A few files appear to be missing for me.

                            Along with
                            Code:
                            #include "ListCallback.h"
                            #include "ProgressCallback.h"
                            #include "7zpp.h"

                            Comment


                            • #15
                              Originally posted by JimStan992 View Post
                              Code:
                              2>A:\Work\Projects\NY_Loft - PC\Plugins\ZipUtility\Source\ZipUtility\Private\ZipFileFunctionLibrary.cpp(4): fatal error C1083: Cannot open include file: 'ListCallback.h': No such file or directory
                              A few files appear to be missing for me.

                              Along with
                              Code:
                              #include "ListCallback.h"
                              #include "ProgressCallback.h"
                              #include "7zpp.h"
                              Is this @ the latest master branch commit? Might be that the pre-requisites are desynced. Let me know which version is broken and I'll have a look.
                              Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                              Comment

                              Working...
                              X