5.8 - Replication changes from 5.7.* proceed with caution!

Development Programming & Scripting Multiplayer & Networking
1

FYI - about upgrades to 5.8 - there’s a few issues with replication. I didn’t see a lot of notes regarding changes, but diving in! (Invalid handles, queries registration, templates not found) etc

#1 - First up - probably not the correct fix…

image
image885×397 60.5 KB

2

What i’m noticing is im spawning two items dynamically, the second one is always corrupted. (I’ve tried different objects). Claude is blaming Epic – im not yet, but just noting what it said :P.

image
image1299×218 19.4 KB

UE 5.8 Bug Report: FastArrayDeltaSerialize delivers corrupted UPROPERTY data to client in Mass Replication bubbles

Summary

After upgrading from UE 5.7.1 to 5.8 (source build), client-side Mass Entity replication receives corrupted FMassEntityTemplateID data via FastArrayDeltaSerialize. The server writes correct data into the FastArray item, but the client deserializes a completely different ConfigGuid and TotalHash, causing SpawnEntities to fail because the template is not registered.

This is reproducible with any Mass replication bubble type (tested with both ProjectileBallistic and CoreCombatant serializers). The issue does not occur on the first replicated item in a given FastArray — it manifests when a second (or subsequent) item is added to the same FastArray while the first item still exists.

Engine Version

  • UE 5.8 source build
  • Reproduced in PIE (Net Mode: Play As Client, 1 server + 1 client in-process)
  • Development Editor configuration, Win64

Reproduction Steps

  1. Create a Mass Entity replication setup using TClientBubbleHandlerBase / FMassClientBubbleSerializerBase / AMassClientBubbleInfoBase (same pattern as Epic’s MassCrowd plugin — MassCrowdBubble.h)
  2. Run PIE in “Play As Client” net mode
  3. On the server, spawn a replicated Mass entity (e.g., a projectile) — it replicates to the client correctly via PostReplicatedAdd → SpawnEntities(TemplateID) — succeeds
  4. While the first entity is still alive in the FastArray, spawn a second entity of the same type from the same UMassEntityConfigAsset
  5. The client receives the second item via PostReplicatedAdd with corrupted FMassEntityTemplateID data — crashes at checkf(EntityTemplate, TEXT("SpawnEntities: TemplateID must have been registered!"))

Evidence

Server-side data (correct)

Both items are added to the FastArray with identical, correct data:

AddAgent SERVER: ArrayIdx=0 TemplateHash=0x84CD22C3E91187A9 GUID=[57e4511444ec3cba4b63bdadd1eba884:0] ReplicationID=1 ArraySize=1
AddAgent SERVER: ArrayIdx=1 TemplateHash=0x84CD22C3E91187A9 GUID=[57e4511444ec3cba4b63bdadd1eba884:0] ReplicationID=2 ArraySize=2

Client-side data (first item correct, second item corrupted)

PostReplicatedAddEntities CLIENT: Idx=0 TemplateHash=0x84CD22C3E91187A9 GUID=[57e4511444ec3cba4b63bdadd1eba884:0]   ← CORRECT
PostReplicatedAddEntities CLIENT: Idx=1 TemplateHash=0xC0577DB9DD9FC505 GUID=[62c6a591c045ec970bb8d7ac40760b2b:0]   ← CORRUPTED

Key observations

  • The ConfigGuid (an FGuid, 16 bytes) is completely different on the client — not a bit flip, not an endianness issue, entirely different bytes
  • The TotalHash is correctly derived from the corrupted ConfigGuid (i.e., TotalHash was recomputed from the wrong GUID, not independently corrupted)
  • The corrupted hash changes between PIE sessions (different wrong GUID each run), ruling out a deterministic serialization offset error
  • All three fields (ConfigGuid, FlavorHash, TotalHash) are UPROPERTY() members of FMassEntityTemplateID and should be fully replicated
  • The first item added to the FastArray always replicates correctly
  • The issue manifests when a second item is added while the first still exists (ArraySize goes from 1 to 2)
  • RemoveAtSwap vs RemoveAt in RemoveAgentImpl makes no difference (tested both)
  • DestroyTemplate is never called (breakpoint verified)

Struct hierarchy (all fields are UPROPERTY)

FProjectileBallistic_FastArrayItem : FMassFastArrayItemBase : FFastArraySerializerItem
├── UPROPERTY(NotReplicated) FMassReplicatedAgentHandle Handle
└── UPROPERTY() FReplicated_ProjectileBallistic_AgentData Agent : FReplicatedAgentBase
    ├── UPROPERTY() FMassNetworkID NetID
    ├── UPROPERTY() FMassEntityTemplateID TemplateID          ← corrupted on client
    │   ├── UPROPERTY() FGuid ConfigGuid                      ← wrong bytes
    │   ├── UPROPERTY() uint32 FlavorHash
    │   └── UPROPERTY() uint64 TotalHash                      ← recomputed from wrong ConfigGuid
    ├── UPROPERTY() FullTransformData
    ├── UPROPERTY() BallisticData
    ├── UPROPERTY() IdentityData
    └── UPROPERTY() DeadReckoningData

Serializer setup (matches Epic’s MassCrowd pattern)

USTRUCT()
struct FProjectileBallistic_ClientBubbleSerializer : public FMassClientBubbleSerializerBase
{
    GENERATED_BODY()

    FProjectileBallistic_ClientBubbleSerializer()
    {
        m_oBubble.Initialize(m_aProjectiles, *this);
    }

    bool NetDeltaSerialize(FNetDeltaSerializeInfo& oDeltaParams)
    {
        return FFastArraySerializer::FastArrayDeltaSerialize<FProjectileBallistic_FastArrayItem,
            FProjectileBallistic_ClientBubbleSerializer>(m_aProjectiles, oDeltaParams, *this);
    }

    // ...

    UPROPERTY(Transient)
    TArray<FProjectileBallistic_FastArrayItem> m_aProjectiles;
};

template<>
struct TStructOpsTypeTraits<FProjectileBallistic_ClientBubbleSerializer>
    : public TStructOpsTypeTraitsBase2<FProjectileBallistic_ClientBubbleSerializer>
{
    enum { WithNetDeltaSerializer = true, WithCopy = false };
};

This is structurally identical to FMassCrowdClientBubbleSerializer in Engine/Plugins/AI/MassCrowd/Source/MassCrowd/Public/MassCrowdBubble.h.

What we ruled out

Hypothesis Ruled out by
Template destroyed between spawns Breakpoint on FMassEntityTemplateRegistry::DestroyTemplate — never hit
Wrong EntityConfig asset Server log shows same asset, same GUID, same hash for both spawns
TotalHash not a UPROPERTY (not serialized) Confirmed UPROPERTY() at MassEntityTemplate.h:94
RemoveAtSwap corrupting array slots Changed to RemoveAt — same result
Cross-replicator handle confusion Fixed separately (bool-returning RemoveEntity callback) — unrelated
Client registry missing the template Template is registered (first spawn succeeds with identical hash)
Unstable ConfigGuid on asset Server logs show stable GUID 57E45114... across all spawns within a session

Current workaround

Downgraded the checkf in UMassSpawnerSubsystem::SpawnEntities(FMassEntityTemplateID, ...) to skip and log a warning when the template is not found. The entity doesn’t spawn on the client that frame, but the server remains authoritative so gameplay is unaffected.

Expected behavior

The client should receive identical FMassEntityTemplateID data to what the server wrote into the FastArray item. All UPROPERTY fields should be correctly serialized and deserialized by FastArrayDeltaSerialize regardless of how many items are in the array.

Environment

  • Windows 11 Pro
  • UE 5.8 source build (upgraded from 5.7.1)
  • Visual Studio 2026 18.7.1
  • DX12 / SM6
  • Mass Entity framework with custom replication (4 bubble types: CoreCombatant, SimpleRepresentation, ProjectileBallistic, HomingObject)