sqlitesupport issues

I’ve included sqlitesupport by compiling it and using the source build for my project - no problems there, but when I go to use it, it’s returning empty values. The statements parse and return a result set with the correct column names, but when I retrieve values from them they’re empty/0. What could be the problem?

Additionally, when i’m inserting data it doesnt actually get inserted. sqlite3_prepare_v2 is returning OK.

Did you download and placed the “amalgamation” files before you build engine with SQLite support module?

Btw, that module is largely incomplete. I have a plugin I built from scratch that doesn’t require engine to be rebuilt. I could share on Marketplace, but then I would have to rundown a code review first to make sure the code adhere to EpicGames plugin requirements.

I built a plugin from scratch mainly because from there I can control which functions compile to Dedicated Server and which ones are automatically stripped from Client packaging, this is main reason did it.

I’d be happy to do that review for you for the sake of getting this damned thing working already :wink: I also think it’s something people would pay for although then you’d probably get the usual “how does it work in blueprint?”.

I think of sharing this after I add automatic serialization for TMap and TSet fields in Blueprints (send these types to and from a database directly);
When I wrote this plugin TMaps and TSets weren’t supported by Blueprints so, in the serializer functions where I extract UProperty values and insert into a SQL db or read them back, there’s no functions to read those types yet which is bad today’s state of the engine.
I would also have to update everything converted to Unreal 4.18+ API, which is very different from before, so I don’t know when I can upload this; I will take a look and let you know :slight_smile:

Here’s a screengrab of the DB file object (I edit SQL Tables and DB Files within Unreal Editor with this plugin enabled):

Any Actor in the game with UProperties of the types and Field Names declared in the DB Table are automatically serialized to the database when I use a “Save” function; it’s like the ‘Save Game to Slot’ system, but doesn’t use that FArchive thingy; I send values directly to the database instead.

Wow you’ve really gone all out! That editor could be adapted to any sql implementation. Write an interface that others can implement to plugin to it and sell it :smiley:

I’ve been working on custom serializer based on Unreal’s reflection system lately…
So today I tried to add my serializer to the SQLite’s UObject and see if, based on those functions, sending out a TSet<> variable to a SQLite database would work.
I made a TSet and populated with some values then sent to the DB, here’s output on DB panel:

It works! :eek:
But for whatever reason Unreal won’t allow me to use a TSet of FRotator or FText tho, I didn’t know the engine doesn’t support those types with TSet instead of TArray…
Because it’s a serializer built from ground up, I also tested with Structs assets (Blueprint Structs) that are marked “SaveGame” tag and it also worked.

Will work on this thingy this next weekend if I don’t have to “crunch” on something else; try to add TSet and TMap support to this SQLite parser.

Oh god an unholy mix of SQL and JSON.

Yes, but only Structs I convert to Json, primitive types are just strings. It’s better than converting to an alien string format or save a big blob of binary data.
Oh and I learned today that newest version of SQLite does the exactly same thing I was doing by myself lol

http://sqlite.org/json1.html

Instead of parsing manually with SQLite 3.9 you’d have to just call the functions they provide on that extension.

That’s pretty cool

Was messing around some more with this today.
I’ve successfully included to the custom serializer support for reading/writing TSets, TMaps, Float Curves, Object Pointers and TArray of Object Pointers…
Took me just some 1500 lines of code :s

Because I’m not using FArchives and I’m not relying on the “<<” operator for serialization, TArray<UObject>* can be saved and loaded properly from a DB row.
I personally love the fact *FRuntimeFloatCurve *serialization to the db row is working, because I use that thing a lot for RPG stuff…
I’ve currently tested it with these types:


UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) int32 IntTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) bool BoolTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) uint8 ByteTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) float FloatTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FText TextTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FColor ColorTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FString StringTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FName NameTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) UObject* ObjectTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FRotator RotatorTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FVector Vector3DTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FString TimeStampTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FVector2D Vector2DTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FCustomStruct StructTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FDateTime DateTimeTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) ECustomEnum EnumTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FLinearColor LinearColorTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) FRuntimeFloatCurve FloatCurveTest;

UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<int32> IntArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<bool> BoolArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<uint8> ByteArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<float> FloatArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<FText> TextArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<FString> StringArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<FName> NameArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<UObject*> ObjectArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<FRotator> RotatorArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<FVector> Vector3DArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<FVector2D> Vector2DArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<ECustomEnum> EnumArrayTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TArray<FMyCustomStruct> StructArrayTest;

UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TSet<int32> IntSetTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TSet<bool> BoolSetTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TSet<uint8> ByteSetTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TSet<float> FloatSetTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TSet<FString> StringSetTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TSet<FName> NameSetTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TSet<FVector> Vector3DSetTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TSet<FVector2D> Vector2DSetTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TSet<ECustomEnum> EnumSetTest;

UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TMap<int32,int32> IntMapTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TMap<bool,bool> BoolMapTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TMap<float,float> FloatMapTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TMap<uint8,uint8> ByteMapTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TMap<FString,FString> StringMapTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TMap<FName,FName> NameMapTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TMap<FVector,FVector> Vector3DMapTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TMap<FVector2D,FVector2D> Vector2DMapTest;
UPROPERTY(Category = "TEST", EditDefaultsOnly, SaveGame) TMap<ECustomEnum,ECustomEnum> EnumMapTest;

Wow, nice. Does it work with non-uobjects too? int64 is pretty important for id’s even though it’s not supported as a UPROPERTY. Also, it’s probably worth throwing FVector4 and FQuat in your round of tests just for fun.

Vector4 and Quat are structs pretty much only used in materials editor.
But they will serialize just fine as a “generic” struct if declared as UProperty.

int64 can’t be supported because my serializer can only “see” types exposed to reflection (UProperty types).

I use TFieldIterator<UProperty>() combined with *Property->ContainerPtrToValuePtr<void>() *to extract or insert property values in/from sqlite db.
That’s not possible without reflection so that will never work on variables that are not UPROPERTY marked CPF_SaveGame flag.

​​​​
Epic gives a lot of useful info about it here:
https://www.unrealengine.com/en-US/b…tem-reflection

Oh, and the “IDs” on db I intentionally force them to be text only, because I have a custom function to generate db IDs that must be FNames.
The id is a composition of Object name, owner, player id or network owner id so I can check if that object recorded actually belongs to a server or to a client or if it’s just a loose object without any network priority…
Otherwise wouldn’t be possible to identify a record of the same character class for different players, for example, loading HP for Player 1 would load for all players if had no such composed IDs in place.

I’ve added to my editor a data section to check entries that are currently recorded into the generated *.db file (:

Oh and in case Blueprint users wonder how complex it is to use this on Blueprints… well the API is too complex. * not really lol *
You have to implement an Interface to the Actor/Component Blueprint then call a bunch of nodes from Interface events, these nodes:

And… Provided your Blueprint has properties named the same of Column names on the Table panel… then that’s all, properties will save/load from db when nodes “DB Save” or “DB Load” are called. :3

This is almost complete…

I’ve implemented DB Tables & Class versioning (you can patch your game and still read SQL tables that record outdated property names).
I’ve implemented threaded save & load mode (you can make the database asset switch to threaded or sync loading with an enum button on the db editor panel).
I’ve implemented automatic calculation of save and load workloads (the plugin automatically reports from 0.00% to 100% the state of save or load operation). This is for feeding loadscreen progress bars :slight_smile:

And… I finished the in-editor SQLite database editor :slight_smile:
We can delete, copy and paste records from a “data preview” panel that can display all records currently stored on the (real) *.db file; this however only works for object/actors that are currently loaded in UEditor to avoid bad editor performance.

Oh, and I built a demo of the threaded system saving and loading around 20 “RPG” properties for 101 characters in the level.
“Attributes” is a gameplay tag, “Weapons” is a TArray<UObject*> (Array of object references) created in Editor, not C++… they can save and load successfully from the database file :slight_smile:
DEMO EXE

Now I just want to make a dynamic loading screen with progress bar as example and I guess I’m done ^^ [HR][/HR]

Fantastic!

Okay, so… I’ve implemented 5ish~ auto-generated load screen modes.

  • No Load Screen: (keeps playing the game as normal, load/save on background).
  • Background: (adds to HUD a Slate widget with progress bar. If “Blur” set to zero).
  • Background Blur: (same of above, but will blur the background by amount you set).
  • Splashscreen: (Progress Bar with a background image. like Fortnite’s load screen).
  • Movie Player: (Progress Bar with a MP4 video on background).

An then I’m done with this, seems pretty stable so far :slight_smile:
Would you like to beta test it?

WARNING: There’s zero documentation beyond UI tooltips on UEditor and demo project!

[HR][/HR]

http://i.imgur.com/MFnwDly.png

http://i.imgur.com/UZpSbkV.jpg

http://i.imgur.com/at2Y4aQ.png

This looks very impressive.

// My QUESTION was already answered by Bruno per E-Mail.

Oh no… I was implementing a solution like this myself… Im wondering if I have to stop now since Im still at 70% overall implemented… and yours got a bit more functionality built into.

Will it take long to be available at MP?

I can’t keep up reading the whole forums for stuff, lol