When serializing UDataTable to json, RowStruct is always serialized as null (understandable, as UScriptStructs need to be created by reflection code, and cannot be saved to json).
In UDataTable::Serialize, there is a piece of code that saves the path name of RowStruct into serialized property RowStructPathName, which is serialized just fine.
// Make sure and update RowStructName before calling the parent Serialize (which will save the properties) if (BaseArchive.IsSaving() && RowStruct) { RowStructName_DEPRECATED = RowStruct->GetFName(); RowStructPathName = RowStruct->GetStructPathName(); }However, in deserialization, RowStructPathName is never used, RowStruct is always loaded as null, therefore LoadStructData loads all data as FTableRowBase, effectively discarding all data.
I think this is trivially fixable by just looking up the RowStruct after deserialization with something like this. Is there are reason this isn’t supported?
if (BaseArchive.IsLoading() && BaseArchive.IsTypeOrArchiveThatWouldntSaveUScriptStructReference() ) { RowStruct = FindObject<UScriptStruct>(RowStructPathName); }
I’m not sure I fully understand the use case here. Exporting to JSON and reimporting should be fully supported and functional in the editor.
What archive type are you using to serialise as JSON? It sounds like the call to Super::Serialize should handle serialising RowStruct as it is a UPROPERTY, unless the archive doesn’t handle serialising pointer types.
I think I see the issue now. It’s getting caught up due to the empty ObjectIndicesMap as you’ve noted. The only other place in the engine that I see that code is in SavePackage2.cpp. I’m not overly familiar with that part of the codebase, so I’m going to hand this case to somebody with more knowledge of that area of the codebase who can advise on this.
As a workaround and potential solution, you may be able to create a new subclass of the JsonArchiveOutputFormatter that overrides Serialize(UObject*& Value) if you wish to change that behaviour. It would complicate the deserialisation and adds assumptions to the formatter, but you may be able to serialise it in another form. Just be aware that there may be unintended issues if the RowStruct has pointers internally as well. SoftPath and SoftPtr will serialise as string safely though.
the code that is intended to fix up the RowStruct class is happening in UDataTable::PostLoad():
if (!RowStructName_DEPRECATED.IsNone()) { UStruct* SavedRowStruct = RowStruct; if (!SavedRowStruct) { SavedRowStruct = FindFirstObjectSafe<UStruct>(*RowStructName_DEPRECATED.ToString()); } if (SavedRowStruct) { RowStructPathName = SavedRowStruct->GetStructPathName(); } else { UE_LOG(LogDataTable, Error, TEXT("Unable to resolved RowStruct PathName from serialized short name '%s'!"), *RowStructName_DEPRECATED.ToString()); } }So the current implementation is expecting PostLoad to be called after serialization, which happens if you load an asset in UE, but not if you manually serialize it like you’re doing.
You could try calling PostLoad() manually, but loading is quite intricate, so there’s a chance this will surface other issues.
Otherwise if your current solution works, feel free to stick to it.
You’re right this code specifically fixes up the change from a string based member to the one using FTopLevelAssetPath.
I’ve talked to someone a bit more familiar with the datatables and they’ve recommended not to roll your own serialization with the JsonArchiveOutputFormatter.
There are utility functions in the UDataTableFunctionLibrary, for example UDataTableFunctionLibrary::FillDataTableFromJSONString that have existing logic to load/save datatables from json and deal with a lot of the pecularities. These use GetTableAsJSON() and CreateTableFromJSONString() internally.
I’d suggest you have a look at those and see if they would already cover your use-case.
My use case was transferring some data from running editor to the Standalone play mode launched from editor. I saved a table to some temporary json file and then loaded it in game. This is the code I originally used:
TUniquePtr<FArchive> OutputAr(IFileManager::Get().CreateFileWriter(*FilePath)); FJsonArchiveOutputFormatter JsonFormatter(*OutputAr.Get()); FStructuredArchive StructuredArchive(JsonFormatter); FStructuredArchiveRecord RootRecord = StructuredArchive.Open().EnterRecord(); Data->Serialize(RootRecord);And then the same thing, except with JsonArchiveINPUTFormatter when loading. This will always serialize the RowStruct as null.
However I since found a workaround - passing a manually constructed ObjectIndicesMap with the RowStruct at some arbitrary PackageIndex (PackageIndex has protected constructor, so I use FromImport):
TMap<TObjectPtr<UObject>, FPackageIndex> ObjectIndicesMap; ObjectIndicesMap.Add(GetRowStruct(), FPackageIndex::FromImport(RowStructPackageIndex) ); JsonFormatter.SetObjectIndicesMap( &ObjectIndicesMap);And then using the InResolveObject of FJsonArchiveInputFormatter to resolve this arbitrary PackageIndex to the correct RowStruct (I know beforehand which RowStruct will be serialized in there).
auto LookupRowStruct = [this](const FPackageIndex& Index) -> UObject* { if (Index == FPackageIndex::FromImport(RowStructPackageIndex) ) { return GetRowStruct(); } return nullptr; };I haven’t found any examples of intended usage of FJsonArchiveInputFormatter. But intuitively what I did here seems very wrong: I am abusing the PackageIndex, and creating essentially a fake ObjectIndicesMap and lookup function. Is there a better way to do this?
Oh, I missed this, probably because the _DEPRECATED suffix, I assumed this was some unused backwards compatiblity thing.
However it doesn’t work unfortunatelly. In UDataTable::Serialize(before PostLoad can be called), LoadStructData is called, and if at that point the RowStruct is not loaded, all the data is deserialized as if it were FTableRowBase - which has no properties, effectively meaning all data in the file is discarded, and the table is filled with FTableRowBases.
Not only that, calling PostLoad() now sets the RowStruct on a table filled with FTableRowBases, which will now be reinterpret_casted to the new struct whenever data in the table is accessed, most likely crashing the program.