Replicating TMaps, what are my options?

Development Programming & Scripting C++
, , , ,
1

I was working on a stat management system to easily allow my gameplay designer an easier method for assigning stats to various classes.

The idea was to have an enum for each datatype which contains the various stats that belong to that data type. Then we have a TMap which uses said enum as a key, and is exposed to blueprints where the defaults could be set.

Having it be a TMap meant it was a little simpler to just include the stats that needed to be there, as well as to check if the actor uses a given stat, and we really liked using this system.

However I didn’t even consider checking before hand, but to our dismay, TMaps don’t replicate. I assumed they would since TArrays replicate by (as I understand it) only updating the data that has changed and needs to be updated. And tbh I thought it may be simpler considering if you remove the first index of an array, you have to sort of shuffle the entire array -1, but with maps you just change the value of any changed keys.

Anyway the point is, as the title says, is there a good workaround for replicating TMaps to clients?
Else is there an alternative that provides similar functionality but allows for replication?

2

The simplest (already classic) way of “replication” is to create two replicated arrays (for keys and values) and change them through special functions that will be similar to TMap.

I read somewhere (or maybe I’m confusing something) that TMap is difficult (impossible) to replicate, because it is cached internally and each time (including for each client) differently, so you can’t just take and transfer a new value over the network.

1 Like
3

agree with @PREDALIEN , although adding you could also put them in a struct.

i wonder though if we can serialize the TMAP to a ByteArray, repnotify that and then deserialize it back, and i wonder how that would perform.

4

I did see a solution where someone used an array of TPairs which I think would make what I’m trying to do a bit easier, but in that solution they opted to replace all values in the map. However this is something that would update frequently enough that I’d like to take performance into consideration.

Since we’re working with maps and not arrays, am I correct in assuming it would be pretty safe to only replicate the key values that change?

5

I think the reason TMap cannot be replicated is it is optimized for key lookup speed, which means the order of the keys potentially changes every time you add/remove/modify a key.

If runtime lookup speed is your primary concern, you could potentially create a duplicated TArray of structs that only exists purely for the sake of replication, and essentially every time you modify your TMap you also modify the TArray, and then on the client when the TArray changes replicate, you update both it and the TMap on the client side. The struct would contain both the map key and its value.

The benefit of this approach is you keep your runtime lookup efficiency, at the cost of double the RAM and a bit of extra CPU when you modify the TMap contents. The network bandwidth would be optimized since only key/value changes would replicate.

6

You could optimize it by just sending over the delta in the TMap (if only parts change).

You would then only have to update incrementally and the data send over the network could be smaller.

So if the sent data key exists in the map then update the value, if it doesn’t, add the value. You could send a bit to describe if it’s a delete or add / update function. If the bit corresponds to delete then remove the entry with the passed over key from the TMap

7

To be fair, I’m not quite as concerned about the network performance, since unreal already only updates the changes from an array. It’s mostly the CPU performance when updating the map values on the client.

I did think about only replicating the “delta” as you put it, but for late joiners it’s going to require 2 arrays to be replicated. Assuming by delta you mean only the values that changed since the last time the array was replicated. If you mean just storing every variable that has changed over the lifetime of the server, then I feel like at some point we’re just back at square one.

Unless there is an overridable function or something that triggers when a new player joins the server, then I can just put every value from the map into the array

8

Do the values per class ever change?
If the stats are an “archetype” then you could replicate an asset reference, where the reference points at some particular instance of something that contains the TMap in question.
If you also keep your stats in a TMap on the player character and/or player controller, I would not actually replicate changes on that map, but instead have a function that calculates the value of a particular stat when it is needed.
Thus, you can replicate “base class” as an asset reference, and then you can replicate all the things that will buff/debuff stats, and each client will calculate the state in question correctly.

9

So if I’m understanding correctly, you’re suggesting instead of replicating the values themselves, I replicate the “modifiers” being applied to a given stat, which can then be computed locally?

10

Correct. Plus, you replicate “what is the base class?” which gives you the initial stats.
The modifiers can all live in an array. (Or more than one array, if you want level improvements to live separately from temporary buffs/debuffs)

11

I think that would probably be the best solution in my case, much appreciated!

12

Actually wait, wouldn’t this still be back to the first issue? I’d still run into the issue of having to iterate through the “modifiers” to update the variable on the client wouldn’t I? And then I’ll not only have the complexity of setting those values, but computing the changes.

Or if you’re suggesting I only pass most recent “modifiers” into this array, I still have the issue of late joiners not receiving all of the changes. So it seems like either way I need a second array, unless I’m not understanding correctly

So far the best solution I can think of is having one “initializer” array, that has all the updated stats replicating. Then a bool to basically use as a “DoOnce” for the first repNotify. Then a second array that’s only meant for changes.

This would cost extra network bandwidth for the “initializer” array though, as well as some memory and a bit of CPU performance of course.

13

(post deleted by author)

14

my approach

bool FMyFragmentPair::NetSerialize(FArchive& Ar, UPackageMap* Map, bool& bOutSuccess)
{
	if (!Key.NetSerialize(Ar, Map, bOutSuccess) || !bOutSuccess)
		return false;
	if (!Value.NetSerialize(Ar, Map, bOutSuccess) || !bOutSuccess)
		return false;
	return true;
}

bool FMyFragments::NetSerialize(FArchive& Ar, UPackageMap* Map, bool& bOutSuccess)
{
	bOutSuccess = false;
	
	if (Ar.IsSaving())
	{
		TArray<FMyFragmentPair> pairs;

		for (const TPair<FGameplayTag, FInstancedStruct>& fragment : Fragments)
		{
			pairs.Emplace(fragment.Key, fragment.Value);
		}
		
		Ar << pairs;

		bOutSuccess = !Ar.IsError();
	}
	
	else if (Ar.IsLoading())
	{
		TArray<FMyFragmentPair> pairs;
		Ar << pairs;

		Fragments.Empty();
		Fragments.Reserve(pairs.Num());
		
		for (FMyFragmentPair& pair : pairs)
		{
			Fragments.Emplace(MoveTemp(pair.Key), MoveTemp(pair.Value));
		}

		bOutSuccess = !Ar.IsError();
	}
	
	return bOutSuccess;
}
15

There are a number of ways to compute the stats.
The simplest possible way to do this, would be to implement a GetStatValue() function.
This function would simply:

  • get base stat value from base class
  • iterate through all active modifiers, which are arrays of {stat, delta} pairs
  • for the modifiers that affect this stat, update the value
  • return the result

This might feel like it’s “a lot of work,” but, honestly, it will probably never show up in a profile of the game, unless you do something excessively expensive with stat values, or have hundreds of units all with modifiable stats.

If you absolutely have to improve it, then you can keep a map of state → calculated value on the client, and use RepNotify for each of the arrays. When you get the notify, set a “stats dirty” flag. In your GetStatValue() function, if the “stats dirty” flag is set, then re-compute all of the stats, and store them in a local (not-replicated) stat map, and clear the “stats dirty” flag. Then always return the value from the stat map.

You could even check this flag in your Tick() function and recompute there if you needed. Note that replication is never tick precise anyway, so any kind of delay is something you’ll have to live with in a networked game anyway.

Another benefit of a system like this is that save games or character checkpoints can be robust. You will never “drift” between what’s actually applied, and what the calculated value is. If you save a character while it has a temporary buff, that temporary buff affect will still be there when you load it back, and re-applied as appropriate. And when it expires, it will no longer be applied.

Btw, you can keep an array of {stat, delta, expiration time} tuples, if you have many temporary stat changes; that might be easier than keeping separate affects that have to remember to remove their temporary stat affects.
Tuples can be any replication-enabled UObject or Struct class.

16

To be fair, the main reason for my concern is our use case involves large waves of enemies that could be afflicted by any number of status effects at once. Not to mention we’re targeting mobile hardware (quest), which thankfully we have a really simple art style that shouldn’t cause much graphical performance issues, but CPU will likely be the bottleneck here.

Although I do like the system you’ve suggested and will probably implement something similar, I still think I may still have to find a solution to reduce the amount of iterations of stat changes. I think I’m still looking for control in marking specific array elements as dirty, or I guess read which array elements are dirty.

I could probably do this within a struct, which I replicate all the map values, as well as a second array with each key value that’s been dirtied. The client could apply the whole maps values on the first go around, then update the deltas for the following notifies. Either the server or the client can periodically try to send/receive instructions to apply the entire array to the map. Or if there are enough stats / unique instances of stats, only certain portions of the map can be updated at a time to ensure synchronization.

I’ve also thought about having multiple arrays that map to different ranges of values, since we’re working with Enums as keys there would only be a fixed range of 0-255. So I could have an array that only holds and replicates keys 0-9 for example. However I couldn’t think of a reasonable implementation

This probably is a bit overboard though, since of course there are other things I can do such as having separate maps for stats that are less critical to be updated, and stats that are more critical, but I just wanted to see what solutions are already out there

17

For some reason the whole situation seems a bit like code smell. Why would you even need to replicate a TMap for status effects?

Status effects are used as modifiers for a characters attributes (buffs / de-buffs). Why not have a static list of effects that can be referenced by the component that does the temporary modification?
It’s not like you will be making new status effects dynamically on-the-fly.
They require art, sound, and logic => not really something you would make dynamically through any type of network fed data.

That way you would have no need to replicate the TMap at all.

It would be build once and local copies would live on the client and server. Then just use the reference to the tap key and and assign a value to the change.

Only this would need to be replicated => map key and value.

18

The TMap would actually be for the stats, not the status effect. But if say multiple enemies are burning and they’re all losing health frequently, no reason to update all stats, right?

But sure, a smarter way to handle that would be to keep it separate from less important stats to be updated. I just wanted to see if there were options since we liked how TMaps made our stats feel more modular.

19

Wouldn’t you have stats assigned to your characters already? Calculated based on for instance an internal level system, maybe some modifiers like a champion variant etc.

These could be stored as data assets or in a data table and also be locally calculated.

Still not seeing why they would be replicated unless you are thinking of making enemies via random generation.

20

Yeah I suppose you’re right, most stats are probably going to be constant anyway, there should probably only be a small group of stats that are even going to be updated during runtime for the enemies. I was definitely putting the cart before the horse here.

I figured “current” values of stats that update frequently would need to be replicated directly in order to avoid desync. I mean I would have updated them locally anyway, but then have them corrected by the server to be sure.

Appreciate all the help despite my admittedly stubborn approach