Weak Map Invalid Key Runtime Error

Please select what you are reporting on:

Verse

What Type of Bug are you experiencing?

Verse

Summary

Using weak_map sometimes throws a Verse runtime error

Steps to Reproduce

LoadPlayerData():void=
        if (not SavedPlayerData[Player]):
            if (set SavedPlayerData[Player] = player_save_data{}) {}  
        set Data = SavedPlayerData[Player] or player_save_data{}

Expected Result

For Verse not to crash

Observed Result

Verse crashes

Platform(s)

all

Verse unrecoverable error: ErrRuntime_WeakMapInvalidKey: Invalid key used to access persistent `var` `weak_map`.

Truncated callstack follows:
player?a_manager:(/projects.epicgames.com/_ac96613d49629c8c4ab549953320cf9a/player_data_manager:)LoadPlayerData (Unknown source) (Source: Unknown(0,0, 0,0))
player_manager:(/projects.epicgames.com/_ac96613d49629c8c4ab549953320cf9a/player_manager:)AddPlayer(:player,:?ForceAdd:logic = ...) (Unknown source) (Source: Unknown(0,0, 0,0))
player_manager:(/projects.epicgames.com/_ac96613d49629c8c4ab549953320cf9a/player_manager:)OnPlayerSpawned(:agent) (Unknown source) (Source: Unknown(0,0, 0,0))
player_manager:(/projects.epicgames.com/_ac96613d49629c8c4ab549953320cf9a/player_manager:)OnPlayerSpawned11(:any):void (Unknown source) (Source: Unknown(0,0, 0,0))


The status of FORT-742803 incident has been moved from ‘Needs Triage’ to ‘Needs More Info’.

Do you only hit this runtime error when the player is not in the session? That’s likely what’s happening here

You likely want to check if Player.IsValid before accessing your weak_map(player,…)

1 Like

Weap map access is already fallable, why wouldn’t this be handled internally? It seems contrary to the philosophy behind Verse to have these kinds of errors just lying around.

I’m getting this issue as well and its Agents being thrown by popup dialogues

It seems to happen randomly but adding sleeps seems to help, could this be some kind of thread safety issue on the server?

I am going to try and double buffer this as a solution - maintain a non weak dictionary and queue saving to the persistent somehow

Failure is used to handle the case where the player is not in the map at all. Failure when accessing a map or weak_map is tied to the key is not in the map.

We runtime error when the player is no longer in the session (e.g IsValid fails) but in the weak_map because this is something we will support in the future. We just don’t yet have the technology built to support it. To do this right, it requires us building distributed transactional memory. We runtime error instead of use failure because runtime error will be forward compatible with us fully supporting this feature in the future. We don’t use failure as a mechanism to indicate an unsupported feature. We fall back to runtime error for that. Failure is something that’s directly observable in Verse, and runtime error is not.

This has been a general implementation strategy for us: there are places in the implementation where we runtime error today but want to support a complete semantics in the future. To prove my point, another example of this is around integer overflow. In the currently shipping VM, ints are represented by 64-bit ints. This means we create a runtime error when arithmetic on ints overflow. However, in complete Verse semantics, ints are unbounded. The new VM we’re working on supports unbounded ints, and will no longer raise runtime errors for some cases where the current VM may raise a runtime error.

1 Like

I really REALLY appreciate the insights here. I also really wish I had known this months ago. So many sleepless nights chasing these game crashes without knowing what was happening.

Now that I understand this I need to immediately go and fix all our maps.

Lastly, the only problem now is that IsActive fails when a player is respawning. So there will be issues with persistence and data updates.

Unless you know another way, it seems like I’m going to have to use a custom WeakMapDelay function in the case of a respawn (versus a player leaving server)?

4 Likes

Are you sure? I think @saamyjoon meant the new IsActive on player that was added with verse persistence, not fort_character IsActive. Makes no sense that it’s not active while respawning.

edit
I saw Axel said the same:
image

Doesn’t make much sense if that’s the case. Do we have to make it suspends with a timeout and hope for the best? Tbh I imagine many creators struggling with implementing it correctly (or even being aware they have do to it).

1 Like

player.IsActive (not fort_character.IsActive) is the method intended for avoiding the runtime error when indexing into a persistent weak_map. It shouldn’t change behavior based on the player respawning (maybe leaving and rejoining the game, but not respawning).

1 Like

This is actually something I’ve been digging into as of late and wondering what exactly is going on behind the scenes with the Playspace and why we may be seeing inconsistencies. Something fundamentally backwards to me is the way the API comments infer that we get a player reference and I believe ties into the larger issue (as this is all I have to go off.)

Playspaces<public> := module:
    # A nested container that scopes objects, style, gameplay rules, visuals, etc. All objects and players in an experience will belong to a fort_playspace. There is typically one `fort_playspace` for an entire experience, though this may change in the future as the platform evolves.
    # 
    # To access the `fort_playspace` for a `creative_device` use `creative_device.GetPlayspace`.
    fort_playspace<native><public> := interface<epic_internal>:

        # Signaled when a `player` joins the `fort_playspace`. Returns a subscribable with a payload of the`fort_character` that entered the `fort_playspace`.
        PlayerAddedEvent<public>():listenable(player)

        # Signaled when a `player` leaves the `fort_playspace`. Returns a subscribable with a payload of the`fort_character` that left the `fort_playspace`.
        PlayerRemovedEvent<public>():listenable(player)

Returns a subscribable with a payload of thefort_character that entered the fort_playspace

Seems indicative of legacy system implementation. I’ve arrived at the conclusion that there must be some fusion of the fort_character and the agent_controller(talking AController uobjects here) causing a lot of the inconsistencies and errors that can be observed.

player.IsActive (not fort_character.IsActive ) is the method intended for avoiding the runtime error when indexing into a persistent weak_map

Indicates that these two things need to be treated separately, which supports class construction and inheritence from a UObject perspective. Typically AController and APawn extend AActor independently and are treated as their own objects, where AController possesses APawn, but based on the way the Playspace has been set up it actually indicates that we get a reference to the PlayerController (extends AController) object via the Fort_Character (extends APawn).

I realize that it may not be accurate to represent UObjects in a 1:1 correlation with Fort_Char and Agent but I have to construct a fundamental framework to understand and apply these concepts since one has not been provided.

The Problem: Fort Character (fort_character) seems to have evolved to represent the character (pawn extension), but originally meant the player_controller and its associated character object, and was all treated as a single entity. This has become problematic as they are no longer addressed as the same object, and now have divergent definitions we can point to indicating such. We are led to believe that Fort_Character is the physical representation as stated in the UObject definitions, but we somehow derrive a Player reference from the Playspace aka fort_playspace through the Fort_Character according to the Playspace API Comments. The Fort_Character’s existence and relationship to the Agent is an external “blackbox” to us. (bare with me…)

With the new Player.IsActive[] indicates that we now properly have the capacity to query the Player Controller’s existence separately from the Fort_Character which allows us to treat these two objects as independent of one another, and accurately reflects the future implementation that I believe is being referenced here.

Where there starts to be a breakdown and where I think the bugs related to this occur from is that the Agent and Fort_Char existence seems to still be treated as a guarantee when they in fact are not:

Reviewing when events are being signaled by your devices and whether or not they contain the appropriate payload (ie: returning agent controller in response to a spawn event, which explicitly handles bringing a Fort_Character into existence) would be a great place to revise and improve the clarity and effectiveness of implemented API functionality.

The Playspace and comments indicate that the entirety of the core of the Playspace functionality may not be clearly or explicitly defined one way or another. It would be great to have someone review internal definitions and functions to ensure nothing was overlooked here, and then update the API comments to reflect the current state by which it is handled.

I know this doesn’t address weakmaps directly but I think that we fundamentally need to look at the Playspace itself and ‘fix the foundation’ in order to suppose anything else as our entire ability to interface with the game is dependent on this objects existence, which is inherently either flawed or misrepresented

1 Like

I imagine player.IsActive is effectively just GetPlayspace().GetPlayers().Find[Player], and the comments about fort_character subscribables are just typos or outdated

1 Like

I suppose then it’s just down to clarifying which objects exist and when and ensuring your checking the validity of the right object for the given function. I think that most of this stuff follows logically, but it takes quite a bit of work to get to these conclusions and they’re still uncertain at best.

Maybe just separating agent events from fort_character events and providing subscribables to both in the API would be an elegant solution? I just think that they’ve been a bit muddied and some distinction here would provide value. While you do want to get an agent reference on spawn once, you generally want a fort_char that’s active as a reference every other time.

Also, wanted to add this to the justification for why i’m suspicious generally about the playspace and whether or not everything is actually utilizing the same systems here. I don’t think that the historical fort_char authorative player controller situation was fully resolved.

Somehow spectator can’t resolve the fort_char camera when calling Respawn() which indicates that it’s failing yet the players definitely playing with their camera.

I think that code was originally written to expect a guaranteed Fort_Char and when the system was updated and failable context was added it allowed for these things to silently fail (at least for us) and they just haven’t ever been fully remedied. In this case it could be that fort_char is now optional and upon elimination the fort_char gets set back to false prior to the next respawn, but Respawn() still has a reference to the previous fort_char which it then uses to spawn for the character without updating the value. Again though, impossible to say with it all being a black box.

Just throwin’ it out there since these things tend to go unaddressed and I find this a rather foundational issue to be having with the platform. We need some sort of solid ground to stand on and make presuppositions from, and this definitely ain’t it.