Runtime DataTable - Import/Export text CSV or Google Sheet at runtime and auto-update structs and UObjects!

JaredTherriault · October 23, 2019, 10:36pm

This plugin will load a text csv or download a public Google Sheet and add parsed values to a matching USTRUCT at Runtime. All data types supported by the built-in UE4 DataTable are supported by this plugin.

Price: $19.99

Link: **https://unrealengine.com/marketplace…time-datatable

Runtime Data Table or “RDT” is a plugin for Unreal Engine 4 that loads a Comma-Separated Values or CSV document at runtime with a USTRUCT’s variables in text form and fills an array of that USTRUCT with the appropriate values, converting text into the appropriate type.

RDT can be likened to the Data Table system built into UE4, but while that system works only at design time, this plugin is made to work while the game is running.

The system works by downloading a public or private Google Sheet or loading a CSV from disk and attempts to fill an array of USTRUCTS with values loaded and converted from the CSV. After the attempt is made, the plugin will call a function defined by the user to signal that the plugin has finished its duties.

Features:

Runtime parsing of Data Tables
Automatic filling of USTRUCTs
Connectivity with Google Sheets - edit values on the fly, no need to reimport CSVs manually
-Export CSV to local text file or Google Sheets (with authentication!)

Code Modules:

easeyCSV: A runtime module that handles CSV data parsing
Runtime Data Table: A runtime module that handles the download/ and parsing,

Number of Blueprints: 22

Number of C++ Classes: 3

Network Replicated: No

Supported Development Platforms: Win64, Mac

Supported Target Build Platforms: Win32, Win64, Android, Mac, iOS (+tvOS for versions 4.25 and 4.26, no longer supported in 4.27 due to Google Sheets Export)

Documentation: Documentation is found here and in the Resources folder.

Demo: A Win32 packaged demo can be found here. Includes a TestStruct.csv (WindowsNoEditor\CSV_Test\Saved) that can be edited with the results showing in-game on-demand when ‘Execute It’ is clicked.

What Is Runtime Data Table and Please Video-Teach Me How to Use This Thing So I Don’t Have To Read

Update Feb 21 2022:

Heads up everyone, the Maui update is coming very soon, likely within two weeks.

The big feature for this update is, of course, the long-requested Write to Sheets.

It’s been a long time coming and was difficult to make work, but it is indeed working and I’m just finishing up with polish for it and adding documentation.

I will be raising the price to $19.99 for this update, so if you haven’t bought in yet do it before the update goes live to lock in the $14.99 price!

Update Feb 24 2022: Maui update is live!

JaredTherriault · August 22, 2021, 6:52pm

4.27 version has just been submitted! Should only be a couple days before it goes live.

JaredTherriault · August 26, 2021, 8:29pm

4.27 version is ready for download!

HenryFabricioPM · September 10, 2021, 3:46am

After received this warning I see that the easyCSV is used inside Runtime DataTable plugin. What I do?

UATHelper: Packaging (Windows (64-bit)):   LogPython: Warning: 'EasyCSV.MakeCsvInfoFromString' and 'EasyCSV.MakeCSV_InfoFromString' have the same name (make_csv_info_from_string) when exposed to Python. Rename one of them using 'ScriptName' meta-data (or 'ScriptMethod' or 'ScriptConstant' for extension functions).
UATHelper: Packaging (Windows (64-bit)):   LogPython: Warning: 'EasyCSV.MakeCsvInfoFromFile' and 'EasyCSV.MakeCSV_InfoFromFile' have the same name (make_csv_info_from_file) when exposed to Python. Rename one of them using 'ScriptName' meta-data (or 'ScriptMethod' or 'ScriptConstant' for extension functions).
PackagingResults: Warning: 'EasyCSV.MakeCsvInfoFromString' and 'EasyCSV.MakeCSV_InfoFromString' have the same name (make_csv_info_from_string) when exposed to Python. Rename one of them using 'ScriptName' meta-data (or 'ScriptMethod' or 'ScriptConstant' for extension functions).
PackagingResults: Warning: 'EasyCSV.MakeCsvInfoFromFile' and 'EasyCSV.MakeCSV_InfoFromFile' have the same name (make_csv_info_from_file) when exposed to Python. Rename one of them using 'ScriptName' meta-data (or 'ScriptMethod' or 'ScriptConstant' for extension functions).
UATHelper: Packaging (Windows (64-bit)):   LogInit: Display: LogPython: Warning: 'EasyCSV.MakeCsvInfoFromString' and 'EasyCSV.MakeCSV_InfoFromString' have the same name (make_csv_info_from_string) when exposed to Python. Rename one of them using 'ScriptName' meta-data (or 'ScriptMethod' or 'ScriptConstant' for extension functions).
UATHelper: Packaging (Windows (64-bit)):   LogInit: Display: LogPython: Warning: 'EasyCSV.MakeCsvInfoFromFile' and 'EasyCSV.MakeCSV_InfoFromFile' have the same name (make_csv_info_from_file) when exposed to Python. Rename one of them using 'ScriptName' meta-data (or 'ScriptMethod' or 'ScriptConstant' for extension functions).

ChrisK · November 4, 2021, 2:51am

Hi,

Thanks for the handy asset. I think it helps dealing with DT at runtime.
But I have a couple of questions.

While testing, I’m trying to call “Construct” from your example project multiple times. It read from csv file and output to another csv file. It works fine. But the problem is that when I call “Construct” again, it appends the same output to the exported file again and again.
I think the problem is that CSVInfo needs to be cleared before I call it again but I can’t seem to clear CSVInfo struct. Can you please help?
Input csv has a column to reference to StaticMesh like this. “StaticMesh’/Game/MESH/tank_01/tank1_cylinder_007.tank1_cylinder_007’”
but when I export the reference, it outputs like this. “ObjectProperty /Game/Inventory/ST_Item.ST_Item:Mesh_41_735FD56B4D0B93C45E82309FC51D3EA1”
How do I keep the output string the same as the input string?
I need to read them back in so that reference has to be the same as the input.

Thanks.

JaredTherriault · November 4, 2021, 4:33am

Hi @ChrisK !

The reason for this is that the array is not emptied before calling update array from csv info. Just call “Empty” on the array before then and this won’t happen. It’s an oversight with the test project - Update array from csv does not clear the array before adding new members while the Google Sheets method DOES clear the input array first. I’ll have that fixed in the next version. In the meantime just clear the array first.
When working with object references, it’s best to use Soft Object Path instead of a hard reference. This is advantageous both with Runtime DataTable and without. With RDT the soft object path will resolve to a string so it will load back in the same way. Hard object references have a different hard path on each engine startup. This isn’t an RDT thing, it’s an engine thing. Another advantage to soft object path is that you don’t have to load the object you’re referencing until you want to. Hard references always stay in memory, soft object paths are just a sign pointing to where the object is and you can decide when to resolve the reference into the object in question.

ChrisK · November 4, 2021, 6:28am

Hi, @JaredTherriault
Thanks for the quick answer.

Yeah, the output result from “Make CSV Info From File” is “CSVInfo” and I tried to both make new CSVInfo before calling and clear the “Map” inside of CSVInfo but both of them didn’t work. Sure, please update the version (to 4.27) once you have a chance.
Sure, I’ll give the soft-object a try. Thanks

ChrisK · November 4, 2021, 7:02am

Regarding #2, there is a way to get the asset reference path from a given(hard) object.
A BP function from Victory plugin and I verified that it works.
I suppose you can use the following when writing a reference to UObject as there is no point in writing hard references.

FName UVictoryBPFunctionLibrary::GetObjectPath(UObject* Obj)
{
	if(!Obj) return NAME_None;
	if(!Obj->IsValidLowLevel()) return NAME_None;
	//~~~~~~~~~~~~~~~~~~~~~~~~~
	
	FStringAssetReference ThePath = FStringAssetReference(Obj);
		
	if(!ThePath.IsValid()) return "";
	
	//The Class FString Name For This Object
	FString str=Obj->GetClass()->GetDescription();
	
	//Remove spaces in Material Instance Constant class description!
	str = str.Replace(TEXT(" "),TEXT(""));
	
	str += "'";
	str += ThePath.ToString();
	str += "'";
	
	return FName(*str);
}

JaredTherriault · November 4, 2021, 5:19pm

The plugin doesn’t write the text that ends up in the CSV, it uses an engine method to stringify everything then arranges the CSV. Then it parses the CSV and uses an engine method to turn it back into a type. I’m not comfortable with the idea of changing specific behaviours that would differ from how Data Tables work, otherwise I would just turn the objects into soft object references automatically. But that isn’t the engine behaviour.

JaredTherriault · November 4, 2021, 5:23pm

Regarding #1, you don’t want to empty the map, you empty the array that’s passed into the update method. Try something like this (please excuse the crudely drawn highlights):

ChrisK · November 5, 2021, 12:51pm

Hi,
Of course, I tried to clear the array but that didn’t work.
CSVInfo keeps growing.

JaredTherriault · November 5, 2021, 4:26pm

Hmm, could you show your blueprint setup? I can compare it against my own and try to play a little ‘spot the difference’

ChrisK · November 8, 2021, 8:40am

Sure, I’m not doing anything special.
I think the problem is that somehow, CSVInfo is not getting cleared each time it runs.

JaredTherriault · November 8, 2021, 10:00pm

Ok, so you are clearing the array for sure, but one thing that could be an issue here is that the variable is replicated. Is this function running on the server? Or from the construction script? If you disable replication on that array, does the problem persist?

One way you can test to see if CSVInfo is being cleared or not is by running a foreach loop on the map keys and printing them out - if you see the keys being duplicated then that means it’s not being cleared. If they don’t duplicate then the issue is with the array.

aramland26 · January 8, 2022, 8:44pm

Hi, I’m new to blueprints, and I’m just trying to implement a google sheet to update this blueprint…I’m having a lot of trouble tho…is it posssible?

JaredTherriault · January 10, 2022, 4:48am

Hi aramland26!

What you’ve shown me here is a normal data table. Have you started using the Runtime DataTable plugin specifically? Could you show me what you have tried using the nodes from the plugin?

aramland26 · January 13, 2022, 7:34pm

Hi! I already almost got everything to work, exept my BP that work with Orientation(N,E,W,S) I need to split the string into booleans so I can make it work with, but I have no idea.

JaredTherriault · January 15, 2022, 3:44am

I’m not sure I follow, can you show me where this information will ultimately go?

Is there any reason that you’re getting the data as string from the parsed csv rather than getting the data directly from the array of structs? I only ask because everyone has their use cases and perhaps I can build a better solution for you.

aramland26 · January 17, 2022, 5:08pm

Sorry, I’m new to blueprints and I’m trying my best, so the only reason I’m using as a string if because it’s the only way that I found to separete the directions ( altough it didn’t work) , the info goes to a widget blueprint that is supposed to turn on and off some color boxes, depending of the orientation that each apartment has. Sorry if I can’t explain myself better

JaredTherriault · January 17, 2022, 7:42pm

So it looks like you have a struct called POI struct that has the information you need for the UI.

When you call “Make CSV Info From String/File” you’re given the CSV Info struct. It looks like what you’re doing here is you’re getting the information as string from the CSV which is a valid way of working with the data, but it’s the hard way

After “Make CSV Info From String/File” call 'Update Array From Csv Info". It will ask you to add an array of structs that match the struct type with your data, i.e. a variable of type array of “POI Struct”. It can be empty, it just has to be an array type variable wrapping “POI Struct”.

When this operation completes, your struct array will be filled with data, one item for each line in the CSV file. You won’t have to get individual cells as string this way, the data will be automatically converted for you. Then you can open up each DirectionStruct inside POI Struct with the correct data.

En español:

Parece que tiene usted un struct llamado ‘POI Struct’ que tiene la información que necesita usted para UI.

Cuando usa usted “Make CSV Info From String/File”, este le da usted un struct llamado “Csv Info.” Parece que está usted tomando la información como string del CSV que es válido pero es mas difícil.

Después de “Make CSV Info From String/File” use usted 'Update Array From Csv Info". Le pide usted de una matriz o ‘array’ de struct que refleja el tipo de struct con sus datos, en efecto una variable de tipo matriz de ‘POI Struct.’ Se puede estar sin miembros la matriz, solo tiene que ser del tipo ‘POI Struct’.

Cuando completa esta operación, su matriz será llenada con los datos, un miembro para cada línea en el documento CSV. No tiene que usted recoger la información como string, los datos se convertirán automaticamente. Entonces abra usted cada ‘DirectionStruct’ al dentro de cada ‘POI Struct’.