I’ve looked around, and just about everything I find related seems to be deprecated, can anyone tell me where to start? Blueprint and C++ would be preferable but I can deal with either or.
So depends on what your needs are…
what do you need it for…
whether you are targeting ios, consoles, or pc…
In general, you can include the single file sqlite amalgamation as part of a plugin. In the past compiling was more annoying, but is fairly straight forward.
TBH Issues like packaging the db, saving at runtime, etc… are a pain.
As an alternative, now I often just use datatable(s) imported from csv. Then at runtime just use various TArray(s) populated from the datable(s).
While datatables are readonly, TArrays are writable, accessible from BP, and easily persisted in savegame.
I’m looking to make an inventory system on PC that can be saved to a server and sent to clients QUICKLY even if there’s A LOT of data (1,000,000 items if possible) I was considering doing it in SQLite, having the SQLite file downloaded from a server to the client and when it’s updated, relay back to the server to update it on the server.
I’m starting to consider relaying MySQL or python or maybe some other C++ program via HTTP in order to dynamically update SQLite
I’ve already established the ability to communicate and update with MySQL, do you think it’d be practical to somehow convert a MySQL DB to CSV?
I’m starting to guess my answer now lies within something similar to this
http://www.convertcsv.com/sql-to-csv.htm
Then converting that into TArray strings per line
make that my persistent inventory and when it’s updated (such as item removed from inventory) I should still be able to remove lines at specific locations and relay the MySQL to update, correct? I’m just wondering now how quickly 1,000,000 rows of items could be called from a SQL database and generated into a CSV file on connect.
If you have a central server and want to send data to clients - probably best to stick with web friendly protocol like https and send data as json.
no need to mess with datatable or csv - just process http respone and update tarray directly
You think the http response would be able to gather say 1,000,000 rows?
What’s the limit on TArrays?
update*
I’m attempting a 10 million row TArray right now, will increase if successful
update*
After successfully generating 10 million rows in all of 5 seconds after launching the game, I decided to give 100 million rows a go - this resulted in about a 15 second load time and considering the type of numbers (extremely ****ing unrealistic to the point everyone thinks I’m crazy) that I’m trying to attain - this is perfect. The idea behind my entire philosophy is that I could generate a NEAR-ENDLESS inventory that will load in under 20 seconds initially and be able to be updated dynamically without any lag - this is perfect. “NEAR-ENDLESS” is a **** TON of objects. I would be satisfied with one million, but I definitely want to have it stress tested beyond that JUST IN CASE. For what I am currently trying to achieve there is only one known replica and it caps out at around 50,000-100,000. I wanted my numbers to hail in comparison - even if they don’t have to.
After some research I’ve also concluded that HTTP protocol has no string size limit and that I MIGHT be able to put a timer on the rate each TArray is added after a certain size causing lag is attained - thus reducing the lag to an absolute zero and allowing user inventories to expand by streaming endlessly.
If you are looking to transmit data between clients then why aren’t you using the built-in networking architecture and serialization framework?
Why are you reinventing the wheel?
It’s what I do.
I’m transmitting a ■■■■ TON of data, that has to be remembered permanently.
Comfortably clocked at 20,000,000 items being feasible, I’m still trying to increase that number.
I’ve build an attribute/inventory system which uses SQLite as a cache running on Unreal Server and then later sync with a MS Server through an external C# server application.
This way is possible for the game to have many instances of a server running, a player can connect to any of these instances, load and save data from SQLite in real-time and still have all their data saved to a final MS Server database from any of the SQLite databases from any of the Server instances running while accurate diffing which attributes are kept from which Server, based on date-stamps attached to each SQLite record.
Each UE4 Server then later read that data from the Server and replicate to connected Clients eligible to receive replicated attributes from players around (using the builtin network replication system). I don’t use any json or any http request stuff; The built UE4 Server with this SQLite cache system now can record a bazillion data every second, the MS Server database then is configured to sync to/from its “real database” every XX seconds.
It’s funny you made this response because that is essentially what I was considering was an external program. Props for actually doing.
Let’s talk about this because I just stumbled upon this method accidentally, now I’m stuck and decided to come look at this post and I’m realizing you were telling me how to do what I’m attempting. Can you explain exactly how to “include the single file sqlite amalgamation as part of a plugin.” I’m currently attempting this and failing in every direction, I’m a complete idiot when it comes to the creation of plugins or even how to correctly include them in a project without messing everything up. So if there’s any way you could give me a complete step by step for dummies or point me in the right direction that would be greatly appreciated!
Bump to keep relevant
Wrote an ios oriented post a while back: http://www.casualdistractiongames.com/single-post/2016/06/03/Using-SQLite-with-Unreal-on-iOS
And just posted this brief update: http://www.casualdistractiongames.com/single-post/2017/10/10/SQLite-With-UE-415and-above
Hope it helps…
We hope this plugin can help you.
Yeah, $40 for something I already made for free?
Maybe I should go release the same thing for $15
Please do that, in all seriousness.
This plugin here: https://www.unrealengine.com/marketplace/usqlite-database has a lot more features and for loading and saving data from actors you don’t even need to write a single SQL sentence. I agree thou that FREE is appealing, but it has so many interesting features and is being heavily maintained by the creator, with updates and added features.
Try this SQLite3 Plugin: