Announcement

Collapse
No announcement yet.

Maps (Data) needs more functions

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    [FEATURE REQUEST] Maps (Data) needs more functions

    To avoid confusion with maps (used in levels as geometry), this is about the maps, used to store data.

    The maps still lack some crucial functions, notably :
    -Make map from 2 arrays
    -Set Map Elem
    -GetKey and GetValue as pure and reference
    -Find returning the index


    Explainations :
    • Make map from 2 Arrays: At runtime, you can't just use Make Map (that actually requires you to input keys and index by hand), and since maps are basically two arrays, this makes sense
    • Set Map Elem: The current only way to modify a map is to remove the index then add it back, and that has quite a lot of performance overhead. You can't just do a getValues then Set array Elem, because getValues doesn't give the reference
    • GetKey and GetValue as pure and reference: Why not ? I know it can break the map if you remove an index in the array without removing the same in the other one, but that shouldn't be an issue, just put a warning on it.
    • Find returning the index: It's a find, and it's two arrays, so when you're using one of the arrays, you know where that is.

    And this is why I'm not currently using maps but two arrays linked together by code, so that kinda sucks.
    Last edited by Moddingear; 12-03-2017, 04:36 AM.

    #2
    A Map is NOT two arrays. Their functionality is very different from arrays.

    Comment


      #3
      Originally posted by Raildex_ View Post
      A Map is NOT two arrays. Their functionality is very different from arrays.
      What I mean by that is that you can achieve exactly the same thing by syncing up two arrays.

      Comment


        #4
        Make map from 2 Arrays: At runtime, you can't just use Make Map (that actually requires you to input keys and index by hand), and since maps are basically two arrays, this makes sense
        That is not true. You can Add elements to a Map dynamically - nothing stops you from using 2 arrays for that. What am I missing here?

        Set Map Elem: The current only way to modify a map is to remove the index then add it back, and that has quite a lot of performance overhead. You can't just do a getValues then Set array Elem, because getValues doesn't give the reference
        That's what Add is for. If the provided key already exists, its value gets overridden.

        GetKey and GetValue as pure and reference: Why not ? I know it can break the map if you remove an index in the array without removing the same in the other one, but that shouldn't be an issue, just put a warning on it.
        Not sure if I follow; Find fetches a value and Contain tells you whether the Key exists.

        Find returning the index: It's a find, and it's two arrays, so when you're using one of the arrays, you know where that is.
        I guess this could be useful in some scenarios, although never felt it was missing. You're using a Map because you do not want to deal with indexes.
        Last edited by Everynone; 12-03-2017, 11:50 AM.

        Comment


          #5
          Map is NOT represented by two arrays in memory. It's represented by some kind of binary tree for faster look-up.

          So, point by point
          - making from two arrays, possible but it'll be the same as you going over every key-value pair adding them one by one - write your own helper function for it and be happy.
          - modifying map by modifying result of GetValues - technically impossible as map is not stored in two arrays, GetValues cycles over binary tree gathering all the data in one array, unless you could have array of references and even then any addition or removal to map may invalidate whole references array due to map nature.
          - getting GetKeys and GetValues as references - same as above, technically impossible as map is not represented by two arrays and this two functions gather values from binary tree.
          - returning index in find - technically impossible as there are NO stable indexes in map, it's **** binary tree, not two arrays inside, indexes may change on a whim after any map modification.

          So, in general, you have to better understand what map is and what it's used for (good example - FName as key and whatever as value) or continue to use two arrays as you do now. Sacrificing binary tree lookup speed for your convenience to have indexes where there are no indexes is not a way to go.

          Comment


            #6
            Originally posted by ProgFZ23 View Post
            Map is NOT represented by two arrays in memory. It's represented by some kind of binary tree for faster look-up.
            It was a simplification.

            Originally posted by Everynone View Post
            That's what Add is for. If the provided key already exists, it's value gets overridden.
            I did not know that, and that's kinda confusing.

            Originally posted by Everynone View Post
            That is not true. You can Add elements to a Map dynamically - nothing stops you from using 2 arrays for that. What am I missing here?
            Adding the elements one at a time is slow and code-heavy

            Originally posted by Everynone View Post
            Not sure if I follow; Find fetches a value and Contain tells you whether the Key exists.
            Having the arrays as pure makes you able to use Set Members in Struct and that's useful. Also, it make it so that you could use SetArrayElem


            Originally posted by ProgFZ23 View Post
            - returning index in find - technically impossible as there are NO stable indexes in map, it's **** binary tree, not two arrays inside, indexes may change on a whim after any map modification.
            The same applies to the arrays, and why not ? You can remove an index before and you're still fine because you know what you removed. When the map remove something, it mirrors everything on the keys and the values, so everything else gets shifted in memory.





            Comment


              #7
              I looked inside Map code in engine sources, just to be sure. If simplified up to ignoring all complicated stuff - internal representation of map is ONE array of Pairs, each Pair holding key and value while all the fancy stuff around that array is used to find needed Pair faster. So when you request array of keys or array of values it just plain iterates over Pairs in array and copies keys or values depending on what was requested.

              Comment


                #8
                Originally posted by Moddingear View Post
                The same applies to the arrays, and why not ? You can remove an index before and you're still fine because you know what you removed. When the map remove something, it mirrors everything on the keys and the values, so everything else gets shifted in memory.
                Not the same - with array you could predict what will change with 100% accuracy, while for Map it completely depends on what is currently inside of Map and can't be clearly predicted without looking at internal state of Map.

                Comment


                  #9
                  Originally posted by ProgFZ23 View Post
                  I looked inside Map code in engine sources, just to be sure. If simplified up to ignoring all complicated stuff - internal representation of map is ONE array of Pairs, each Pair holding key and value while all the fancy stuff around that array is used to find needed Pair faster. So when you request array of keys or array of values it just plain iterates over Pairs in array and copies keys or values depending on what was requested.
                  ok. But that's the C part of it.

                  Comment


                    #10
                    Originally posted by ProgFZ23 View Post
                    Not the same - with array you could predict what will change with 100% accuracy, while for Map it completely depends on what is currently inside of Map and can't be clearly predicted without looking at internal state of Map.
                    But since it's an array in C++, it's like removing an index ?

                    Comment


                      #11
                      Originally posted by Moddingear View Post

                      But since it's an array in C++, it's like removing an index ?
                      Nope, it's sparse array which could have "holes" in it, that's why you can't predict what will happen when you add new element without looking inside internal part of Map - will it be added to the end or will it fill one of the holes?

                      Originally posted by Moddingear View Post

                      ok. But that's the C part of it.
                      And that C part fully dictates what could be done and what can't be done in blueprints. You can't represent array of 2-component structs as two arrays and expect them to be capable to still influence structs in initial array. At least without witchcraft.
                      Last edited by ProgFZ23; 12-03-2017, 04:13 AM.

                      Comment


                        #12
                        Originally posted by ProgFZ23 View Post
                        Nope, it's sparse array which could have "holes" in it, that's why you can't predict what will happen when you add new element without looking inside internal part of Map - will it be added to the end or will it fill one of the holes?
                        Ok, that explains why. So, no more pure arrays or anything with arrays. I've got it.

                        Comment


                          #13
                          Well, even if it were pure array inside, it'll not change the fact that it's array of structs and so can't be represented by two separate arrays without loosing ability to influence data in initial array, effectively making having indexes completely useless.
                          And, by the way, you don't have access to sparse arrays in Blueprints if I remember correctly.

                          And another "by the way": Blueprints performance is far from C++ performance so if you're really concerned with performance - go for C++ code replacing Blueprints wherever possible. Unless you deploy to a platform that don't support C++ projects, but then it's completely different story.

                          Comment


                            #14
                            I don't get why some of you don't want to have an utility function (a zip function essentially) for making a map out of two lists. It's pretty common all around.

                            Comment


                              #15
                              Originally posted by jonimake View Post
                              I don't get why some of you don't want to have an utility function (a zip function essentially) for making a map out of two lists. It's pretty common all around.
                              Exactly, I'm asking for QoL improvements to maps.

                              Comment

                              Working...
                              X