Announcement

Collapse
No announcement yet.

[PLUGIN] USQLite

Collapse
X
  • Time
  • Show
Clear All
new posts

    [SUPPORT] [PLUGIN] USQLite

    What is "USQLite":

    This is my implementation of SQLite API for Unreal Engine 4.
    This is one of most complex and useful Unreal Engine extensions I've built so far.
    This system empowers you to save and load any data to and from SQLite Databases in Unreal Engine 4 without forcing you to write a single line of SQL Code
    (or Blueprint Spaghetti).
    If you're developing a Game that heavily relies on persistent data that must be reliably saved and loaded all the time, such as RPG Games, your best bet is implementation of a Database such as SQLite to record Player's progress, instead of relying on binary or text files to save your Game...

    But learning SQL programming may become a daunting and lengthy process for you to achieve that goal.
    With this system in place, you can create reliable Databases for your Games, and never care less about SQL syntax.


    Making use of a powerful custom serializer engine, based on Unreal Engine's Code Reflection System, this tool is capable of helping you to do amazing and unique data persistence in runtime that otherwise would be impossible.
    For example: Unreal Engine's "SaveGame" system can't easily save Actor References and restore them to your Properties in real time, but with this system you can:
    • Execute Multi-Threaded SQLite Functions!
    • Create & Edit Database Assets in Unreal Editor!
    • Auto Generate SQL Code through Property Reflection!
    • Setup Property Versioning to support old Game Versions!
    • Save & Load Object References (Pointers) and easily restore them!
    • Save & Load data without conflicts across multiple streamed Levels!
    • Save & Load Actors or Component References and even Arrays of References!
    • Save & Load any kind of Struct as well as basic types like Ints, Strings, Vectors, etc!
    • Save & Load from Background Thread, while players are sill interacting with Game World!
    • (Optional) Progress Bar System can accurately report loading status without freezing the Game!
    • (Optional) HUD System can automatically generate and show Loadscreens when saving or loading the Game.

    Real-Time Auto-Generated SQL Code for Properties:

    Auto-Generated Loadscreens:




    Easy To Use Multi-Threaded API:
    Create SQLite Database Assets in Unreal Editor:
    Search, Delete, Copy & Paste Database Records within Unreal Editor:







    .
    [DEMO PROJECT]:
    [NOTES]:
    Last edited by BrUnO XaVIeR; 04-27-2018, 01:36 PM.
    | Finite State Machine | Auto-Save Plugin | USQLite Plugin | Object-Pool Plugin | Sound-Occlusion Plugin | Anti-Cheat Plugin | Property Transfer Tool |

    #2
    [API REFERENCE]
    USQLite Class:



    Load All Data

    Reads all Data from SQLite.db file on disk and apply loaded Properties to ALL serialized Objects in Game World.
    Optional functionality, such as Loadscreens, may be executed if properly setup by Developer on host Classes.

    Save All Data

    Writes all Data from Runtime Database's Records into the SQLite.db file on disk.
    Optional functionality, such as Loadscreens, may be executed if properly setup by Developer on host Classes.

    Add Command To QUEUE : SAVE

    Adds a SQL Execution String based on this Database's Table Columns that can Serialize Target Object's Properties into SQL DB File.

    Add Command To QUEUE : LOAD

    Adds a SQL Execution String based on this Blueprint's Table Columns that can Serialize Target Object's Properties from SQL DB File.

    Immediate Command: SAVE (Object)

    Executes a SQL Command based on this Database's Table Columns that can Serialize Target Object's Properties into SQL DB File.
    This SQL Command will bypass Save Queue, executing to the DB immediately, progress won't be tracked.
    Excessive use of Immediate Save may cause concurrency issues and Database locks, so use this wisely!

    Immediate Command: SAVE (Component)

    Executes a SQL Command based on this Database's Table Columns that can Serialize Target Component's Properties into SQL DB File.
    This SQL Command will bypass Save Queue, executing to the DB immediately, progress won't be tracked.
    Excessive use of Immediate Save may cause concurrency issues and Database locks, so use this wisely!

    Immediate Command: SAVE (Actor)

    Executes a SQL Command based on this Database's Table Columns that can Serialize Target Actor's Properties into SQL DB File.
    This SQL Command will bypass Save Queue, executing to the DB immediately, progress won't be tracked.
    Excessive use of Immediate Save may cause concurrency issues and Database locks, so use this wisely!

    Immediate Command: LOAD (Object)

    Executes a SQL Command based on this Database's Table Columns that can Serialize Target Object's Properties into SQL DB File.
    This SQL Command will bypass Save Queue, executing to the DB immediately, progress won't be tracked.
    Excessive use of Immediate Save may cause concurrency issues and Database locks, so use this wisely!
    This will not generate any Progress Reports or Interface Events.

    Immediate Command: LOAD (Component)

    Executes a SQL Command based on this Database's Table Columns that can Serialize Target Component's Properties into SQL DB File.
    This SQL Command will bypass Save Queue, executing to the DB immediately, progress won't be tracked.
    Excessive use of Immediate Save may cause concurrency issues and Database locks, so use this wisely!
    This will not generate any Progress Reports or Interface Events.

    Immediate Command: LOAD (Actor)

    ​​​​​​​Executes a SQL Command based on this Database's Table Columns that can Serialize Target Actor's Properties into SQL DB File.
    This SQL Command will bypass Save Queue, executing to the DB immediately, progress won't be tracked.
    Excessive use of Immediate Save may cause concurrency issues and Database locks, so use this wisely!
    This will not generate any Progress Reports or Interface Events.

    Generate SQL from OBJECT : INSERT

    ​​​​​​​Generates a SQL Execution String for Target Object based on this Blueprint's Table Fields that can be used to serialize Target Object's Properties into this SQL DB File.

    Generate SQL from OBJECT : UPDATE

    ​​​​​​​Generates a SQL Execution String for Target Object based on this Blueprint's Table Fields that can be used to update Target Object's Properties into this SQL DB File.

    Generate SQL from OBJECT : SELECT

    Generates a SQL Execution String for Target Object based on this Blueprint's Table Fields that can be used to read Target Object's Properties from SQL DB File.​​​​​​​

    Generate SQL from OBJECT : DELETE

    Generates a SQL Execution String for Target Object based on this Blueprint's Table Fields that can be used to find and delete Target Object's Properties from SQL DB File.​​​​​​​

    Generate SQL from COMPONENT : INSERT

    ​​​​​​​Generates a SQL Execution String for Target Component based on this Blueprint's Table Fields that can be used to serialize Target Component's Properties into this SQL DB File.

    Generate SQL from COMPONENT : UPDATE

    ​​​​​​​Generates a SQL Execution String for Target Component based on this Blueprint's Table Fields that can be used to serialize Target Component's Properties into this SQL DB File.

    Generate SQL from COMPONENT : SELECT

    ​​​​​​​Generates a SQL Execution String for Target Component based on this Blueprint's Table Fields that can be used to read Target Component's Properties from SQL DB File.

    Generate SQL from COMPONENT : DELETE

    ​​​​​​​Generates a SQL Execution String for Target Component based on this Blueprint's Table Fields that can be used to find and delete Target Component's Properties from SQL DB File.

    Generate SQL from ACTOR : INSERT

    ​​​​​​​Generates a SQL Execution String for Target Actor based on this Blueprint's Table Fields that can be used to serialize Target Actor's Properties into this SQL DB File.

    Generate SQL from ACTOR : UPDATE

    Generates a SQL Execution String for Target Actor based on this Blueprint's Table Fields that can be used to serialize Target Actor's Properties into this SQL DB File.​​​​​​​

    Generate SQL from ACTOR : SELECT

    Generates a SQL Execution String for Target Actor based on this Blueprint's Table Fields that can be used to read Target Actor's Properties from SQL DB File.​​​​​​​

    Generate SQL from ACTOR : DELETE

    ​​​​​​​Generates a SQL Execution String for Target Actor based on this Blueprint's Table Fields that can be used to find and delete Target Actor's Properties from SQL DB File.

    Additional References:
    ​​​​​​​
    Complete List of Blueprint Functions:

    ​​​​​​​
    Last edited by BrUnO XaVIeR; 04-25-2018, 09:42 AM.
    | Finite State Machine | Auto-Save Plugin | USQLite Plugin | Object-Pool Plugin | Sound-Occlusion Plugin | Anti-Cheat Plugin | Property Transfer Tool |

    Comment


      #3
      [HOW-TO]

      How to Setup a Database:

      • Right-Click Asset Browser, go to 'Synaptech -> SQL Database' to create a new DB Asset.



      • Open new DB Asset, edit your Table inserting Column Names and Column Types at will.



      • Adjust optional settings as you wish, such as Table Versioning or Loadscreens.



      • On one of your Core Game Classes, such as Game Mode or Game Instance, setup a new SQLite Reference.



      • Set newly created Reference to point at your new DB Asset. You don't have to instantiate the DB Asset, point to it directly.
      • From this Reference you can now call any "[DATABASE]" member functions that fits your Game's architecture.


      Last edited by BrUnO XaVIeR; 04-25-2018, 11:24 AM.
      | Finite State Machine | Auto-Save Plugin | USQLite Plugin | Object-Pool Plugin | Sound-Occlusion Plugin | Anti-Cheat Plugin | Property Transfer Tool |

      Comment


        #4
        [HOW-TO]
        How to Setup a Class to be Serialized to and from USQLite Databases:


        • The 'Easy Way': Re-parent your Class to be a Child of 'SQL Serializable' Actor/Component/Object Class.



        • The 'Flexible Way': Add the 'SQL Interface' to your Class.
        • Then implement its Core Event Functions. (Example provided in Demo Project)




        That's all, your Class is now saving and loading to and from DBs!


        .
        How to Serialize Class Properties:


        • Your Class must have the 'SQL Interface' implemented, like described above, to save Property Values into a Database.
        • Follow that basic requirement, a Property needs to meet three characteristics in order to be picked up by the DB Serializer:
        • Must be marked 'Save Game' tag.
        • Must be named after a Column Field you have defined in your DB Asset's Table.
        • Both Property Type and Column Type must match. DB's Columns governs Types the SQL Serializer modify.

        So if you have a 'Level' Column of Type Integer in DB Table, SQL Code Generator searches in your Blueprints for a Property named 'Level' of Type Int.
        When using Versioning, you can version control Property Names, but not Property Types.



        .
        How to Setup Versioning:

        • You can have multiple SQL Tables inside the same Database recording the same Data.
        • For example: Game_DevTable, Game_Debug, Game_Release, etc.
        • You can create and control which Table your Database is using in the 'Versioning' section of your DB Asset.
        • For Game Classes, you can setup Versioning per Class per Property. You can retarget deprecated Property Names, but not mismatch Property Types:



        .
        How to Setup the Load-Screen System for USQLite Databases:


        • Reparent your Base HUD Class to be a Child of 'SQL LoadScreen HUD' Class:


        Done!



        Last edited by BrUnO XaVIeR; 04-25-2018, 11:46 AM.
        | Finite State Machine | Auto-Save Plugin | USQLite Plugin | Object-Pool Plugin | Sound-Occlusion Plugin | Anti-Cheat Plugin | Property Transfer Tool |

        Comment


          #5
          [FAQs]
          Which Types of Properties are supported for auto-generation of SQL Code?

          These Properties have been tested, used and proven to work with USQLite's SQL Code Generator:
          Code:
          UPROPERTY(SaveGame) bool
          UPROPERTY(SaveGame) int32
          UPROPERTY(SaveGame) uint8
          UPROPERTY(SaveGame) float
          UPROPERTY(SaveGame) FName
          UPROPERTY(SaveGame) FText
          UPROPERTY(SaveGame) FColor
          UPROPERTY(SaveGame) FString
          UPROPERTY(SaveGame) FVector
          UPROPERTY(SaveGame) FRotator
          UPROPERTY(SaveGame) FDateTime
          UPROPERTY(SaveGame) FVector2D
          UPROPERTY(SaveGame) FLinearColor
          UPROPERTY(SaveGame) FRuntimeFloatCurve
          UPROPERTY(SaveGame) FGameplayTagContainer
          
          UPROPERTY(SaveGame) TArray<bool>
          UPROPERTY(SaveGame) TArray<int32>
          UPROPERTY(SaveGame) TArray<uint8>
          UPROPERTY(SaveGame) TArray<float>
          UPROPERTY(SaveGame) TArray<FName>
          UPROPERTY(SaveGame) TArray<FText>
          UPROPERTY(SaveGame) TArray<FString>
          UPROPERTY(SaveGame) TArray<FRotator>
          UPROPERTY(SaveGame) TArray<FVector>
          UPROPERTY(SaveGame) TArray<FVector2D>
          
          UPROPERTY(SaveGame) TSet<UObject*>
          UPROPERTY(SaveGame) TSet<bool>
          UPROPERTY(SaveGame) TSet<int32>
          UPROPERTY(SaveGame) TSet<uint8>
          UPROPERTY(SaveGame) TSet<float>
          UPROPERTY(SaveGame) TSet<FName>
          UPROPERTY(SaveGame) TSet<FString>
          UPROPERTY(SaveGame) TSet<FVector>
          UPROPERTY(SaveGame) TSet<FVector2D>
          
          UPROPERTY(SaveGame) TMap<bool,bool>
          UPROPERTY(SaveGame) TMap<int32,int32>
          UPROPERTY(SaveGame) TMap<uint8,uint8>
          UPROPERTY(SaveGame) TMap<FName,FName>
          UPROPERTY(SaveGame) TMap<float,float>
          UPROPERTY(SaveGame) TMap<FString,FString>
          UPROPERTY(SaveGame) TMap<FVector,FVector>
          UPROPERTY(SaveGame) TMap<FVector2D,FVector2D>
          
          UPROPERTY(SaveGame) UObject*
          UPROPERTY(SaveGame) TSet<UObject*>
          UPROPERTY(SaveGame) TArray<UObject*>
          UPROPERTY(SaveGame) TMap<FName,UObject*>
          UPROPERTY(SaveGame) TMap<UObject*,UObject*>
          Additionally, any custom user-defined Structs or Enums are supported.
          Including Structs created as Blueprint Assets in Editor.

          TMaps of Object Pointers only support <FName,UObject*> or <UObject*,UObject*>.
          Arrays/Sets/Maps of Object Pointers are only supported when members of an Object/Component/Actor. Not member of Structs.
          Last edited by BrUnO XaVIeR; 04-25-2018, 08:21 AM.
          | Finite State Machine | Auto-Save Plugin | USQLite Plugin | Object-Pool Plugin | Sound-Occlusion Plugin | Anti-Cheat Plugin | Property Transfer Tool |

          Comment


            #6
            This will be super usefeul man. I was implementing one myself until I heard you was doing this, which made me completely give up, since I knew yours would be a lot more complete solution than what I was targeting for and I need this so much! Thank you!

            For those who don't know, SQLite is one of the most used database in a single file solution. I myself use it since 1997 and it is a portable solution and can fit in any platform out there. Many products out there in the market uses it embedded as solution from known players: Microsoft, Google, among others in things like the Messenger, Skype, GoogleDrive, etc
            Nilson Lima
            Technical Director @ Rigel Studios Ltda - twitter: @RigelStudios
            Art is a state of Spirit

            Join us at Discord:
            https://discord.gg/uFFSEXY
            UE4 Marketplace:
            Cloudscape Seasons

            Comment


              #7
              Originally posted by BrUnO XaVIeR View Post
              This package will be FREE for you to download during its first week of release on Unreal Marketplace!
              The plugin is gonna be free for the first week?

              Comment


                #8
                Wow. This is fantastic! I've been waiting for an SQLite plugin that had the ability to not only read but write data. I am deeply excited for its release!

                Side question - I was planning on using Savior for my save system, would you say that this technically replaces it if you just saved everything to a DB then? Or would you suggest using both in combination? I.E. Items (DB), Economy (DB), player state (health, location (Savior)), and like NPC corpses(Savior)? Just curious since you made both
                Current Project - *shush shush* Soon. Soon.

                Comment


                  #9
                  Originally posted by abxyxx View Post

                  The plugin is gonna be free for the first week?
                  Yes.
                  Because many people have messaged me and they cannot afford going through Marketplace's payment requirements. So I'm giving these devs starting up, an opportunity to download the system before I "go commercial" ^^

                  I would totally make it free perpetually, but I want to make Marketplace publishing my job someday, I cannot do that without selling something
                  ​​​​
                  | Finite State Machine | Auto-Save Plugin | USQLite Plugin | Object-Pool Plugin | Sound-Occlusion Plugin | Anti-Cheat Plugin | Property Transfer Tool |

                  Comment


                    #10
                    Extremely keen for this! I assume this is what we were talking about in DM a little while ago. Hopefully this is workable as a save system for my single player game!
                    Impromptu Games|dev blog|twitter|itch.io store|Patreon
                    Impromptu Procedural Ladders|Impromptu Procedural Handrails|Impromptu Procedural Stairs
                    |Impromptu Fire Propagation|InFlux Example Game|Impromptu Vector Field Painter

                    Comment


                      #11
                      Originally posted by Arixsus View Post
                      Wow. This is fantastic! I've been waiting for an SQLite plugin that had the ability to not only read but write data. I am deeply excited for its release!

                      Side question - I was planning on using Savior for my save system, would you say that this technically replaces it if you just saved everything to a DB then? Or would you suggest using both in combination? I.E. Items (DB), Economy (DB), player state (health, location (Savior)), and like NPC corpses(Savior)? Just curious since you made both
                      This is "overkill" just to be a save system.
                      You can write something close to 50.000 Actor's Properties to disk per second when using this.
                      * and I mean not all for once, I mean Sqlite can open then write and save the file on disk then close that file that many times per second on PC platforms. *

                      Also it's more geared towards freedom of use for engineers. There's no such a thing as Slots to save and etc, or "Save Game to Slot" functions, etc.
                      You can make a persistent Platformer that just call "Immediate Save" functions the whole Game or you could go as far as build a whole MMO on top of this, serializing SQLite data to a MS Server (I have done that myself with the help of time stamps on Tables).

                      The Savior Plugin is different. It is specifically designed to use the binary ".sav" file format which is supported even on weird Console platforms such as the "Nintendo Switch" for example.

                      This is much more suitable when you're developing something Server based since it has security measures to avoid breaches when Game is packaged in Client mode. Not saying you CAN'T use it as a SaveGame system... I'm just saying you're killing an ant with a tomahawk if you do it
                      Last edited by BrUnO XaVIeR; 04-26-2018, 02:54 AM.
                      | Finite State Machine | Auto-Save Plugin | USQLite Plugin | Object-Pool Plugin | Sound-Occlusion Plugin | Anti-Cheat Plugin | Property Transfer Tool |

                      Comment


                        #12
                        Originally posted by BrUnO XaVIeR View Post
                        ...which is supported even on weird Console platforms such as the "Nintendo Switch" for example.
                        Nice.

                        Also, as an example Conan Exiles use SQLite to store their persistent data, which could be anywhere up to million entries (considering all the player built structures and inventory items).
                        George Rolfe.
                        Technical Coordinator at Orbit Solutions Pty Ltd.

                        Comment


                          #13
                          Originally posted by BrUnO XaVIeR View Post

                          This is "overkill" just to be a save system.
                          You can write something close to 50.000 Actor's Properties to disk per second when using this.
                          * and I mean not all for once, I mean Sqlite can open then write and save the file on disk then close that file that many times per second on PC platforms. *

                          Also it's more geared towards freedom of use for engineers. There's no such a thing as Slots to save and etc, or "Save Game to Slot" functions, etc.
                          You can make a persistent Platformer that just call "Immediate Save" functions the whole Game or you could go as far as build a whole MMO on top of this, serializing SQLite data to a MS Server (I have done that myself with the help of time stamps on Tables).

                          The Savior Plugin is different. It is specifically designed to use the binary ".sav" file format which is supported even on weird Console platforms such as the "Nintendo Switch" for example.

                          Thia is much more suitable when you're developing something Server based since it has security measures to avoid breaches when Game is packaged in Client mode.
                          Not saying you CAN'T use it as a SaveGame system... I'm just saying you're killing an ant with a tomahawk if you do it
                          Interesting! I basically want to use it as a savegame system because of this:
                          For example: Unreal Engine's "SaveGame" system can't easily save Actor References and restore them to your Properties in real time, but with this system you can
                          and I'm fed up enough with not being able to do that that I'd probably nuke the ant.
                          Impromptu Games|dev blog|twitter|itch.io store|Patreon
                          Impromptu Procedural Ladders|Impromptu Procedural Handrails|Impromptu Procedural Stairs
                          |Impromptu Fire Propagation|InFlux Example Game|Impromptu Vector Field Painter

                          Comment


                            #14
                            Yes, I understand why they did it the way it is; but they could've separate more Editor-Things from Gameplay things.
                            For example, I was going to add support to save and load Array/Set/Maps of Object* references from within Blueprint Structs... but deep down in limbo of Unreal Engine's source code there is statement saying something like:

                            "If this theng I think it is is really what it is and it's inside a User Struct... then just explode everything!"
                            So I have to scrap it, even though my code would work there, but the engine will intentionally cause a crash if I do it ^^
                            | Finite State Machine | Auto-Save Plugin | USQLite Plugin | Object-Pool Plugin | Sound-Occlusion Plugin | Anti-Cheat Plugin | Property Transfer Tool |

                            Comment


                              #15
                              Guys..
                              Epic Games just released it on Marketplace!
                              Go get it, if this is useful to you, then please comment and RATE!!***

                              * Help me impress the boss!
                              * Help me impress the ladies ^

                              Oh, the link >> https://www.unrealengine.com/marketp...qlite-database



                              NOTE: Free until MAY 4th 2018!
                              | Finite State Machine | Auto-Save Plugin | USQLite Plugin | Object-Pool Plugin | Sound-Occlusion Plugin | Anti-Cheat Plugin | Property Transfer Tool |

                              Comment

                              Working...
                              X