Hi,
I’ve hit a wall while trying to write a custom NetSerialize method.
What I’m basically doing is serializing some primitive values and a texture. The primitive values are serialized with the << operator of FArchive and the texture is serialized with UTexture2D::Serialize(FArchive&). Well, I try to serialize it this way, but I hit an assert on runtime:
Bad archive positions for bulkdata.
StartPos=-1 EndPos=-1
What am I doing wrong? Did anyone successfully serialized a texture this way?
Hi! Is this texture is from Content folder or is it generated dynamically during your game? The first case can be saved by asset path, the second - by TArray (binary representation of bitmaps)
It’s a dynamic texture (I use USceneCaptureComponent2D to render the scene to a texture).
I’ll try to send it as TArray as you suggest, but I’m afraid that it’ll result in a similar error to the one I got while trying to send the texture like this
auto& textureBulkData = MyTexture->PlatformData->Mips[0].BulkData;
Ar.Serialize(textureBulkData.Lock(LOCK_READ_ONLY), textureBulkData.GetBulkDataSize());
textureBulkData.Unlock();
LogNetPartialBunch: Error: Attempted to send bunch exceeding max allowed size. BunchSize=2097167, MaximumSize=65536
The error from your last comment tells that Ar.Serialize cant work with more data than65536 bytes at once. Several steps to experiment are
So I tried doing this with a smaller texture and it actually worked. Then I tried to chunk it but I’m still getting the message
LogNetPartialBunch: Error: Attempted to send bunch exceeding max allowed size. BunchSize=2097167, MaximumSize=65536
as if you couldn’t serialize 1024 bytes at once in NetSerialize
Here is the code
uint8* data = (uint8*)textureMip0.BulkData.Lock(LOCK_READ_ONLY);
uint64 numOfBytesSent = 0;
uint64 numOfBytesLeftToSend = textureSize;
uint16 chunkSize = 0;
while (numOfBytesLeftToSend)
{
if (numOfBytesLeftToSend > MAX_CHUNK_SIZE)
{
chunkSize = MAX_CHUNK_SIZE;
}
else
{
chunkSize = numOfBytesLeftToSend;
}
Ar.Serialize(data + numOfBytesSent, chunkSize);
numOfBytesLeftToSend -= chunkSize;
numOfBytesSent += chunkSize;
}
textureMip0.BulkData.Unlock();Ok, I managed to do it. Here is the code.
bool FMyStruct::NetSerialize(FArchive& Ar, class UPackageMap* Map, bool& bOutSuccess)
{
if (Ar.IsSaving())
{
auto textureMip0= MyTexture->PlatformData->Mips[0];
int32 textureWidth = MyTexture->GetSizeX();
int32 textureHeigth = MyTexture->GetSizeY();
int8 textureFormat = MyTexture->GetPixelFormat();
Ar << textureWidth;
Ar << textureHeigth;
Ar << textureFormat;
Ar.AttachBulkData(nullptr, &textureMip0.BulkData);
}
else
{
int32 textureWidth, textureHeight;
int8 textureFormat;
Ar << Quality;
Ar << textureWidth;
Ar << textureHeight;
Ar << textureFormat;
MyTexture = UTexture2D::CreateTransient(textureWidth, textureHeight, (EPixelFormat)textureFormat);
FTexture2DMipMap& textureMip0 = MyTexture->PlatformData->Mips[0];
Ar.DetachBulkData(&textureMip0.BulkData, true);
MyTexture->UpdateResource();
}
return true;
}
Two things that are missing from this snippet which I will implement next is
Compress the texture/convert it to a lighter format, like JPEG
Use a well-known and supported pixel format, instead of transferring it over the network.
Thank you very much for the solution! I am trying to do the same, but for some reason it seems like the Ar.AttachBulkData does not modify my FArchive at all. I assume the method might only be supported by some archives.
I’m using FBufferArchive and its size isn’t modified at all with this command, even though the textureWidth etc. do get saved in it correctly. I do check BulkData size beforehand, and the data is definitely there, it just doesn’t get attached.
Which archive type do you pass as parameter into this function in your implementation? Should it be FMemoryWriter?
I had run into an issue using the code provided by woookie2na2, as the AttachBulkData method did not do anything to the FArchvie for some reason.
I ended up using Texture->PlatformData->Mips[0].BulkData->Serialize(DataArchive, nullptr) instead of both the AttachBulkData and DetachBulkData.
I don’t know if it works for really big textures, I didn’t test that as it isn’t needed for my use case. Just wanted to mention this as an alternative solution, in case people run into the same issue.
So the full code is as follows:
bool SaveTextureToArchive(FArchive& DataArchive, UTexture2D* Texture)
{
int32 TextureWidth = Texture->GetSizeX();
int32 TextureHeight = Texture->GetSizeY();
int8 TextureFormat = Texture->GetPixelFormat();
int8 CompressionSettings = Texture->CompressionSettings;
int8 MipGenSettings = Texture->MipGenSettings;
uint8 IsSRGB = Texture->SRGB;
FUntypedBulkData* BulkDataToSave = &Texture->PlatformData->Mips[0].BulkData;
DataArchive << TextureWidth;
DataArchive << TextureHeight;
DataArchive << TextureFormat;
DataArchive << CompressionSettings;
DataArchive << MipGenSettings;
DataArchive << IsSRGB;
BulkDataToSave->Serialize(DataArchive, nullptr);
return true;
}
UTexture2D* LoadTextureFromArchive(FArchive& DataArchive) {
int32 TextureWidth, TextureHeight;
int8 TextureFormat, CompressionSettings, MipGenSettings;
uint8 IsSRGB;
FByteBulkData* BulkDataToFill;
// Get texture info from archive
DataArchive << TextureWidth;
DataArchive << TextureHeight;
DataArchive << TextureFormat;
DataArchive << CompressionSettings;
DataArchive << MipGenSettings;
DataArchive << IsSRGB;
// Create new texture and assign the needed settings to it
UTexture2D* Texture = UTexture2D::CreateTransient(TextureWidth, TextureHeight, (EPixelFormat)TextureFormat);
Texture->CompressionSettings = (TextureCompressionSettings)CompressionSettings;
Texture->MipGenSettings = (TextureMipGenSettings)MipGenSettings;
Texture->SRGB = IsSRGB;
BulkDataToFill = &Texture->PlatformData->Mips[0].BulkData;
BulkDataToFill->Serialize(DataArchive, nullptr);
Texture->UpdateResource();
return Texture;
}
I was using this for saving/loading files, but it can be used for network serialization too I suppose. Just in cases here is the file IO functions, which are using the above functions:
UTexture2D* LoadTextureFromFile(const FString& FullFilePath)
{
TArray<uint8> DataLoadedFromFile;
// Perform checks, return null if something goes wrong
if (!(FFileHelper::LoadFileToArray(DataLoadedFromFile, *FullFilePath))) {
DataLoadedFromFile.Empty();
return nullptr;
}
if (DataLoadedFromFile.Num() <= 0) return nullptr;
FMemoryReader FromBinary = FMemoryReader(DataLoadedFromFile, true);
FromBinary.Seek(0);
UTexture2D* Texture = LoadTextureFromArchive(FromBinary);
FromBinary.FlushCache();
DataLoadedFromFile.Empty();
FromBinary.Close();
return Texture;
}
bool SaveTextureToFile(const FString& FullFilePath, UTexture2D* Texture)
{
FBufferArchive DataToSaveToBinary;
FMemoryWriter DataSavingArchive(DataToSaveToBinary, true);
SaveTextureToArchive(DataSavingArchive, Texture);
if (DataToSaveToBinary.Num() <= 0) return false;
if (FFileHelper::SaveArrayToFile(DataToSaveToBinary, *FullFilePath)) {
DataToSaveToBinary.FlushCache();
DataToSaveToBinary.Empty();
return true;
}
else {
DataToSaveToBinary.FlushCache();
DataToSaveToBinary.Empty();
return false;
}
}This plugin perfectly solves this problem, and runtime Screenshot functionality：
Screenshot as Texture2D in Code Plugins - UE Marketplace (unrealengine.com)
In my case I couldn’t make it work with BulkDataToSave->Serialize (using UE5.4) either.
This is what worked for me (I have an FArchive Ar; defined elsewhere):
Saving
FTexture2DMipMap& TextureMip0 = Texture->GetPlatformData()->Mips[0];
FByteBulkData& BulkData = TextureMip0.BulkData;
FBulkDataBuffer<uint8> BulkDataBuffer = BulkData.GetCopyAsBuffer(BulkData.GetElementCount(), true);
TArrayView64<uint8> BulkDataView = BulkDataBuffer.GetView();
int32 ElementCount = BulkData.GetElementCount();
TArray<uint8> ColorData(BulkDataView.GetData(), ElementCount);
Ar << ElementCount;
Ar << ColorData;
Loading
int32 ElementCount;
Ar << ElementCount;
TArray<uint8> ColorData;
Ar << ColorData;
TArrayView64<uint8> BulkDataView(ColorData.GetData(), ElementCount);
Texture = UTexture2D::CreateTransient(TextureWidth, TextureHeight, static_cast<EPixelFormat>(TextureFormat), TextureName, BulkDataView);
Texture->CompressionSettings = static_cast<TextureCompressionSettings>(CompressionSettings);
Texture->SRGB = bIsSRGB;
I hope this saves some hours of trial&error to somebody