Blueprint Map "find" node that gets value as reference possible? Same as "Get (ref)" for arrays

[k_a_s_s_k_a_s_s]

Is there any reason that the “find” node for maps in Blueprint only returns copies?

For many use cases changing the value directly is necessary, such as setting a member in a struct / vector / any other non-atomic type stored as a value for some key in a map variable. In the same way as there are “Get (Copy)” and “Get (Ref)” for TArrays.
Otherwise, an entirely new struct has to be created, and added with an “add” node, which is far from ideal. For one, it isn’t memory efficient, as the whole struct has to be constructed and saved somewhere else first. Also, it makes the blueprint graph less readable by cluttering it with unnecessary nodes, variables, sequences.

For example, some code like this will introduce no changes at all to “struct variable”, as it operates on the output of the “find” node, which seems to return a copy of the actual value:

https://answers.unrealengine.com/storage/temp/293670-capture.png

It would be way better to be able to access the value directly. Is there some reason for that not to be possible?

Especially confusing why no such node exists since a non-const “find” exists in TMapBase.h in C++:

https://answers.unrealengine.com/storage/temp/293682-capture2.png

I tried to make a custom K2 Node that would do something like this by looking in BlueprintMapLibrary.h and K2Node_GetArrayItem.h, but can’t seem to figure out how the Get (ref) array element node is created at all, and how a value of a UProperty can be returned by ref in an output pin of a node.

Is there anyone here who is familiar with these topics?

[BrUnO_XaVIeR]

The problem is the way the FIND node was programmed.

---

You can get the value of the struct as a ref like this (FVariantType is a custom *BlueprintType *that I created for myself to store anything from like ints, floats, arrays, rotators, transforms, etc):

UFUNCTION(Category="Variables|Variant", BlueprintCallable, CustomThunk, meta=(NotBlueprintThreadSafe, CustomStructureParam="AnyType", DisplayName="Set Variant = <?>",  CompactNodeTitle="SET<?>"))
    static FVariantType VAR_SetVariantValue_ANY( UPARAM(Ref)FVariantType &Variant, UProperty* AnyType );

DECLARE_FUNCTION(execVAR_SetVariantValue_ANY) {

    P_GET_OBJECT_REF_NO_PTR(FVariantType,Variant);

    Stack.StepCompiledIn<UProperty>(NULL);
    void* ValueAddr = Stack.MostRecentPropertyAddress;
    UProperty* AnyProp = Cast<UProperty>(Stack.MostRecentProperty);

    if (AnyProp==nullptr) {Stack.bArrayContextFailed=true; return;}

    P_FINISH;

    P_NATIVE_BEGIN;
      *(FVariantType*)RESULT_PARAM=___SetVariantValue_ANY(Variant,AnyProp,ValueAddr);
    P_NATIVE_END;
}

static FVariantType ___SetVariantValue_ANY(FVariantType &VAR, UProperty* ANYProp, const void* ValuePtr);

FVariantType UGenericLibrary::___SetVariantValue_ANY(FVariantType &VAR, UProperty* ANYProp, const void* ValuePtr) {

    const bool IsInt = ANYProp->IsA(UIntProperty::StaticClass());
    const bool IsFloat = ANYProp->IsA(UFloatProperty::StaticClass());
    const bool IsString = ANYProp->IsA(UStrProperty::StaticClass());
    //... etc


    if (IsInt) {
        UIntProperty* _Int = CastChecked<UIntProperty>(ANYProp);
        if (ValuePtr) {VAR.SetValue(_Int->GetPropertyValue(ValuePtr));}
        else {UE_LOG(LogTemp,Warning,TEXT("{ ___SetVariantValue_ANY }:  Caller context should reference a target Wildcard.
Unable to set Variant VALUE!"));}
    } else

    if (IsFloat) {
        UFloatProperty* _Float = CastChecked<UFloatProperty>(ANYProp);
        if (ValuePtr) {VAR.SetValue(_Float->GetPropertyValue(ValuePtr));}
        else {UE_LOG(LogTemp,Warning,TEXT("{ ___SetVariantValue_ANY }:  Caller context should reference a target Wildcard.
Unable to set Variant VALUE!"));}
    } // else etc etc...
}

[HR][/HR]
The key stuff you have to understand there is ***P_GET_OBJECT_REF_NO_PTR *** and the way ***(FVariantType)RESULT_PARAM= *** is declared inside the P_NATIVE_BEGIN body.
That is, FVariantType which is my custom struct then would be your own struct type that you want to modify.

[k_a_s_s_k_a_s_s]

@BrUnO_XaVIeR

The problem is the way the FIND node was programmed.

You can get the value of the struct as a ref like this
[HR][/HR]
The key stuff you have to understand there is ***P_GET_OBJECT_REF_NO_PTR *** and the way ***(FVariantType)RESULT_PARAM= *** is declared inside the P_NATIVE_BEGIN body.

Wow, thank you so much for such an in-depth answer!

I am not deeply familiar with deeper levels of Unreal’s C++ code base and how UProperties work “under the hood”, so trying to find a macro like P_GET_OBJECT_REF_NO_PTR was quite hard, especially since there is zero documentation about it :')

I will try to make a **Find (a ref) **node now and will post an update soon! The challenging part for me now is making the inputs and outputs wildcards, but hopefully there’s enough I can learn from the existing “Find” node to make that happen.

[k_a_s_s_k_a_s_s]

@BrUnO_XaVIeR

You can get the value of the struct as a ref like this (FVariantType is a custom *BlueprintType *that I created for myself to store anything from like ints, floats, arrays, rotators, transforms, etc):

After some time trying it out, realized there is also a [FONT=courier new]P_GET_TMAP_REF(KeyType,ValueType,ParamName) macro. However it does require known types of key and value, while the node should work with any value/key type, so I am not sure it it’s possible to use it in a wildcard node - there seems to not ever be a complete type in the function defintion for the native [FONT=courier new]Map_Find, instead they are passed as [FONT=courier new]int32& and somehow evaluated later with [FONT=courier new]Cast<UMapProperty>.

Plus another question I can’t find an answer to: Is there also a way to return a value by ref in a K2 node? Can’t seem to figure it out, any [FONT=courier new]& parameters in the function definition seem to be returned by value - the pin is circular and modifying that value doesn’t update it in the map.

[BrUnO_XaVIeR]

I don’t know about that, never tried to do it before.
that one I posted previously I got it working after a lot of trial and error and crashes ^^

[PacaPaw]

Were you able to make the Find(by ref) node?

[k_a_s_s_k_a_s_s]

@PacaPaw

Were you able to make the Find(by ref) node?

Hey! Unfortunately not, although I’ve realized it wouldn’t be possible to make with just CustomThunk, because there is no way to define the output as a reference. I think the way to do it would be through a custom K2Node class, the basics of the process can be found here: Create Custom K2 Node For Blueprint - Epic Wiki

Although this requires making an additonal engine module and declaring it as a dependency, from what I’ve seen.

[AngeIV]

I just ended up making a custom update function. that stores value in a local variable, then modify it and update. I set by key (gameplay tag for me)

[DarkiRuiz]

@k_a_s_s_k_a_s_s

After some time trying it out, realized there is also a [FONT=courier new]P_GET_TMAP_REF(KeyType,ValueType,ParamName) macro. However it does require known types of key and value, while the node should work with any value/key type, so I am not sure it it’s possible to use it in a wildcard node - there seems to not ever be a complete type in the function defintion for the native [FONT=courier new]Map_Find, instead they are passed as [FONT=courier new]int32& and somehow evaluated later with [FONT=courier new]Cast<UMapProperty>…

I’m interested in this because I’ve been looking for something similar, but after reading what you posted I wonder…

From what (little) I’ve gathered about CustomThunk functions, that part before [FONT=courier new]P_FINISH is meant to be used to identify the pin type, so couldn’t you do something like… [FONT=courier new]P_GET_TMAP_REF(FProperty,FProperty,ParamName) (since all pins are “properties”), and later identify the types? Not sure if that would actually work.

@k_a_s_s_k_a_s_s

Plus another question I can’t find an answer to: Is there also a way to return a value by ref in a K2 node? Can’t seem to figure it out, any [FONT=courier new]& parameters in the function definition seem to be returned by value - the pin is circular and modifying that value doesn’t update it in the map…

Er, isn’t “returning a value by ref” kind of incongruous here? I’m talking from my knowledge of C++ limited to mainly UE4 itself, so maybe what I say it’s possible in plain C++ but the reflection system doesn’t support it. Otherwise I suppose you’d simply need to make the return type of the function itself by ref, something like this:

 UFUNCTION(BlueprintCallable, CustomThunk, meta = (MapParam = "TargetMap", MapKeyParam = "Key")
int32& Map_FindRef(const TMap<int32, int32>& TargetMap, const int32& Key); 

[BrUnO_XaVIeR]

Best thing would be creating a custom function that replaces all the actions from “Find” node onwards and simply return nothing.

When returning “value” types, a node will always create a copy then return it, because the “output data” is always stored in a new address in memory (ValueAddr is something else of same type).

You could use ContainerPtrToValuePtr() to change the data directly into the ValuePtr instead of trying to return a ref to it.

[DarkiRuiz]

@BrUnO_XaVIeR

Best thing would be creating a custom function that replaces all the actions from “Find” node onwards and simply return nothing.

When returning “value” types, a node will always create a copy then return it, because the “output data” is always stored in a new address in memory (ValueAddr is something else of same type).

Right, I’ve been checking this stuff lately and realized that, so it’s not possible to “return” values by ref with a plain UFUNCTION even with CustomThunk, you’d probably need a custom K2Node. The thing is that I’ve also checked how the Array Get node does this, and… It’s kinda annoying because from what I’ve gathered it’s deeply buried within the engine’s blueprint compiler code.

So, you need to use a K2Node, but you gotta code your own NodeHandlingFunctor and write the whole process as if it was assembly language.

But wait, there is more. Since the NodeHandlingFunctor still relies on UFUNCTIONS for custom actions, you can’t actually do the thing anyways. The actual by-ref assignment seems to be done by the compiler itself, which specifically targets arrays. I haven’t been able to follow up from there but I’m sure it isn’t possible without modifying the engine source.

@BrUnO_XaVIeR

You could use ContainerPtrToValuePtr() to change the data directly into the ValuePtr instead of trying to return a ref to it.

Wouldn’t that technically be the same as the Map Set node? We want the ref so we can work with specific map entries without having to be calling that function anytime we do changes to them.

This would be particularly useful when you’re also working with data nested within structs or several levels of map abstraction.

For example, I’ve got somewhere an object with a property containing a map of structs. This struct type contains two arrays, which isn’t even too complex compared to some wild sh*t that I’ve seen in the actual engine code.

At the moment, to modify these arrays I gotta:

1. Do a Map Find.
2. Copy the value in a local struct variable.
3. Copy the struct’s array in another local array variable.
4. Do an Array Get.
5. Set the local array’s element.
6. Update the local struct variable with the modified local array.
7. Update the map with the modified local struct.

Imagine now you gotta do this on a loop. In the other hand, if I could access the data by ref I could just:

1. Do a Map Find.
2. Break the result and do an Array Get.
3. Set by ref.

Even looping, it would simplify a lot of work here.

…

In the end, what I did to get around this (in what is probably a very hacky way) was to make a struct that contains a pointer to the property and a raw pointer to its value, and pass it around as if it was the value by ref.

Then I made some variants of the array, set and map functions, but which return this struct instead of the actual value, pointing to the element I want to retrieve/modify.

Finally, I included a function to get a copy of the struct data, and another to set the pointer’s values under the hood.

They probably aren’t very safe, but hey, somehow they work and my computer is still in one piece.

[BrUnO_XaVIeR]

@DarkiRuiz

What I did to get around this was to make a struct that contains a pointer to the property and a raw pointer to its value, and pass it around as if it was the value by ref.

Yes, I am doing the same thing to pass arrays/maps/sets to C# scripts.
I convert their addresses to an int, store in a struct and send to C#.

Epic would probably kick me in the face for that… but hey, it works:

UPTRINT IntPtr = reinterpret_cast(vHandle);

[DarkiRuiz]

Well that’s basically what they do with the reflection system, so they can’t complain, we’re just copying them. <_<

anyways, I sorta gave up on the idea because honestly, I was trying to do a quick and dirty fix, but I found myself wasting too much time and making something too complex for what I needed. Maybe after I’m done with my project I’ll take a look and see if I can come up with a plugin that does this properly.

In the meantime, it would be nice if they added support for this sort of stuff officially, to be honest. I know blueprints are meant to be kinda “begginner-ish”, but there are things that are simpler to do in BPs.

Another example might be stuff like BP smart pointers. That would be really cool, although I understand the world of trouble it might ensue.

[eobet]

@k_a_s_s_k_a_s_s

In the same way as there are “Get (Copy)” and “Get (Ref)”

I would vote for this as well (but Epic is never going to see this or do anything about it, it seems).

I have to do a silly dance with Map Find and Map Add now too…

[JustThatRio]

why can’t you just use the find then add the value back to whatever map you used it on?? that replaces the value of the key your adding to the map which only requires modifying that specific item without the map being fully rebuilt behind the scenes

[k_a_s_s_k_a_s_s]

What you are suggesting achieves the needed end result, but with a lot of unnecessary computations, compared to C++ code that would achieve the same.

Creating an entirely new struct means allocating memory for it, and then copying it over to the map when adding. For smaller structs (i.e. several ints packed for convenience), this is tolerable. Some other structs can be thousands of bytes long, and manipulating just a specific several bytes in them shouldn’t mean re-creating thousands.

[PootisLife]

Okay, I have kind of achieved returning a struct by reference, but it is extremely hacky and not something you could make into a single K2 Node… I basically made a temporary array, used some hackery to pass it by reference into a C++ function that sets it’s ArrayNum to 1 and it’s data pointer to the data you want struct you want passed (had to use a bunch of illegal techniques to do this). Then with the hacked array I send that into the Get (a ref) array node and get the 0th index which will return the struct by reference. But now what you have to do is you actually have to re-nullify the array otherwise when the destructor is called it will crash because it will try and free the memory that was not legally allocated so you have to send the temporary array back into another function to set its ArrayNum back to 0 and set it’s data to NULL. This technically works but the fact that after you get the struct by reference and do whatever you want to do with it and then you have to nullify the array makes it so I could not compact this into a macro because you have to reset it. All the functions implemented a CustomStructureParam so this can be used on any type. This was probably a useless solution lmao. From what I have seen from the K2_Nodes Select and Get (a ref), it seems like there is pretty much no way to return a struct by reference without either some hackery like I did or changing Engine code…

[Daxur_Studios1 (Daxur_Studios)]

Can someone tag an Unreal Developer who may fix this, this is such an important feature!

[eobet]

So I needed this again.

I have lots of pieces of data, which I need to categorize in two ways:

1. Map of all actions involving the data.
2. Map of all targets involving the same data.

Because I need to be able to quickly get answers to “which targets has this data been involved with” and “what actions has this data been involved with”, and if I could have the same data in two different maps this would be easy.

But turns out, it’s still not easy.

So, again, a stupid workaround is to choose to only answer the second question, and then save a Set of answers to the first question with every piece of data, and then every time you want to ask that second question, you have to iterate over all the data and all the Sets.

[Chacobo]

Just wanted to add a +1 here. All these data-driven and ECS voices need actual infrastructure backup.