Announcement

Collapse
No announcement yet.

Landscape performance issue

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Landscape performance issue

    Hi dear Epic,


    I'm editing this post to give a summary.

    The problem as described best by Chosker.

    Originally posted by Chosker View Post
    are there engines out there capable of blending 16 texture 'sets' (diffuse, normal, etc) in a single terrain? surely.
    are there engines out there capable of blending 16 texture 'sets' processed in each and every pixel? I would surely doubt it.

    ...UE4's landscape material system heavily tends towards the second
    The current landscape texturing workflow in UE4 isn't suitable for open world projects. The performance cost per landscape layer goes massively high when compared to other engines. To a point where we're limited to 4-5 layers if we wan to achieve production quality while maintaining acceptable performance. That's while other engines out there i.e Cryengine 2/3/5 or Frostbite 2/3 benefit from up to 16 layers without massive performance drops since many years ago and have made their documentations available.
    We've come down to a number of features that can improve the performance. (You know better, of course).w

    1) Add ability to work with texture arrays into out of box engine version, including tool to create array from texture assets.
    2) Add dynamic branching into material editor.
    3) Overhaul weightmaps.( ex. allowing 16 bit weightmaps, storing layer ID rather than layer weight, and ideally allowing you to customize terrain painting brush to the point where you could manually specify the way to encode custom data)
    4) Expose terrain heightfield / normal map to material editor( You should be able to sample those in terrain's pixel shader. Weight maps too btw.)
    5) Add a dedicated terrain color map.
    6) Untie physical surface ID from terrain weightmaps and allow to bake surface ID from terrain material.
    7) Virtual texturing.

    Cheers to Deathrey for the contribution.

    We're currently "extremely optimizing" our content, but would like to have some improvements behind the scenes so we can develop our open world projects, add more layers and calculations without downgrading the visual fidelity to 2007 era.

    Quotes from the thread:

    Originally posted by Deathrey View Post
    The question is why UE4 landscape system is designed in a way to bump into this limitation, forcing sacrifices in visuals, when this bottleneck can and should be avoided ?
    Originally posted by Drav View Post
    I have a pretty good idea why. Other game engines are using texture arrays, so the costs of sampling multiple textures are much less.
    Originally posted by Deathrey View Post
    Anyone could potentially say that any studio can overhaul it as per their needs. That is true. The reality however is, that at least 3 shipped, well-known and widely marketed UE4 titles, that are known to me, have performance issues which can be attributed to landscape texturing system in one way or another.
    Originally posted by IronicParadox View Post
    ...So that's with six layers on. Any more and my computer will crash because I run out of ram for shader compiling.
    Originally posted by Sitrec View Post
    I'm having similar issues but not as severe. I'm also using a 8x8 map, with tons of layers (although probably only about 6-10 at most per component and they are not too complicated). Just wanted to show that there are more people that would like to see some improvement on this in hope that Epic decides to look into it if enough people ask for it.
    Originally posted by Maximum-Dev View Post
    12.58ms or 79 FPS at 1080p on GTX 970, i7 4790K.
    Material is 4 layers entirely, blended using a splat map.
    Normal map is channel packed with Masks compression.
    And no artistic features involved other than distance fade of textures.
    No gameplay, 3D art assets, A.I, Networking, Particles, foliage etc.
    Originally posted by Deathrey View Post
    at least 3 shipped, well-known and widely marketed UE4 titles, that are known to me, have performance issues which can be attributed to landscape texturing system in one way or another.
    Originally posted by ZenDavis View Post
    There are other engines out whose featuresets we can compare and contrast. If someone else does something better, it's fair to request those features in Unreal. Suggesting a workaround to reach 25 fps is insane.
    Originally posted by HeadClot View Post
    Giving this a bump as this affects me too.
    Originally posted by Maximum-Dev View Post
    We all know a lot of the Kite Demo's performance drop was just due to large textures and high tri counts, but the bigger chunk of the performance was the landscape layers.
    Originally posted by Deathrey View Post
    The need for default landscape improvement is not project specific, but is common for any open world project, hence this thread exists.


    Please improve it. I'll provide any sort of demonstration or anything you need.

    Thanks.
    Last edited by Maximum-Dev; 05-03-2017, 12:33 PM.

    #2
    Giving this a bump as this affects me too.

    Comment


      #3
      I have a pretty good idea why. Other game engines are using texture arrays, so the costs of sampling multiple textures are much less.

      Comment


        #4
        Originally posted by Drav View Post
        I have a pretty good idea why. Other game engines are using texture arrays, so the costs of sampling multiple textures are much less.
        You can use texture arrays in UE4, just have to manually add this and compile the engine: https://github.com/EpicGames/UnrealEngine/pull/2340

        I'm using them, and they are working great
        Last edited by John Alcatraz; 02-26-2017, 07:27 PM.
        Easy to use UMG Mini Map on the UE4 Marketplace.
        Forum thread: https://forums.unrealengine.com/show...-Plug-and-Play

        Comment


          #5
          If texture arrays are the reason other engines can have as many layers but UE4 is limited to ~5 layer before +50% performance is dropped then why is Epic not using texture arrays already?

          Comment


            #6
            Texture arrays are functionally a better alternative to texture atlases in regards to terrain texturing, but in the essence, they are prone to same limitations.
            Realistically, you cannot blend more that 3-4 sets of textures(by set I mean BaseColor, Normal and Masks).

            To my knowledge in both Cryengine and Frostbite you can go pretty wild with landscape layers.
            Last time I've checked CE it was 16 layers per terrain, 1 layer per vertex, essentially allowing you to blend up to 3 layers per triangle. Still pretty limiting, but if you ask me, incomparably better than 3 layers per huge component. And you are storing layer id per vertex, not weight of each layer. All that at expense of not being able stretch the blend region for more than vertex spacing.
            As for FB, judging from the paper available, it is not much different from UE4 when it comes to terrain layer blending. I think unjustified performance difference comes primary from lack of flow control.

            why is Epic not using texture arrays already?
            I bet because current terrain philosophy does not really benefit from arrays.

            Comment


              #7
              I'd just love to have better blending between the layers. Either somehow decouple the blend mask resolution from landscape res or allow us to somehow modulate the transition between the layers. Height blend is a good start but it's not that good at blending stuff.

              Comment


                #8
                Originally posted by Deathrey View Post
                Texture arrays are functionally a better alternative to texture atlases in regards to terrain texturing, but in the essence, they are prone to same limitations.
                Realistically, you cannot blend more that 3-4 sets of textures(by set I mean BaseColor, Normal and Masks).


                Last time I've checked CE it was 16 layers per terrain, 1 layer per vertex, essentially allowing you to blend up to 3 layers per triangle. Still pretty limiting, but if you ask me, incomparably better than 3 layers per huge component. And you are storing layer id per vertex, not weight of each layer. All that at expense of not being able stretch the blend region for more than vertex spacing.
                As for FB, judging from the paper available, it is not much different from UE4 when it comes to terrain layer blending. I think unjustified performance difference comes primary from lack of flow control.
                I bet because current terrain philosophy does not really benefit from arrays.
                I've used CE for so long, but honestly never needed to go beyond 10-12 layers, so even if it's limited to 16 layers that's still pretty impressive, given even at 10-12 layers I never had a performance drop. I know that in Battlefield 3 (Frostbite 2) they had usually 10-12 layers per map too (and that was on 2011 hardware) now 5 years passed we're still struggling with even 4 layers on our terrain which is quite disappointing for non-paragon games.

                Comment


                  #9
                  Originally posted by jonimake View Post
                  I'd just love to have better blending between the layers. Either somehow decouple the blend mask resolution from landscape res or allow us to somehow modulate the transition between the layers. Height blend is a good start but it's not that good at blending stuff.
                  you can make custom transitions by using the layer weight nodes to output a 0-1 mask (instead of your actual layer textures), which you can then bias/modify and then plug into a lerp alpha (which will blend between your actual layer textures). it gets more expensive and since you can't use the layer switch it gets more cumbersome, but its doable
                  Follow me on Twitter!
                  Developer of Elium - Prison Escape
                  Local Image-Based Lighting for UE4

                  Comment


                    #10
                    Originally posted by Maximum-Dev View Post
                    I've used CE for so long, but honestly never needed to go beyond 10-12 layers, so even if it's limited to 16 layers that's still pretty impressive, given even at 10-12 layers I never had a performance drop. I know that in Battlefield 3 (Frostbite 2) they had usually 10-12 layers per map too (and that was on 2011 hardware) now 5 years passed we're still struggling with even 4 layers on our terrain which is quite disappointing for non-paragon games.
                    The default way of texturing landscapes in UE4 is suffering from one huge drawback. Inconsistent framerates. Even if you manage to put your level artist in a cage, and make him plan a level in a such way, that any terrain component does not have more than 3 layers(which is much harder than it sounds), you would still get high framerate fluctuations due to components with only 1 layer rendering faster. Jumping framerate at times can be worse than stable low framerate. FB sorts this out with a virtual texturing.

                    Comment


                      #11
                      I suppose you have overkilling textures sizes and complex shaders, i use 8 layers without problem.

                      Comment


                        #12
                        Originally posted by erWilly View Post
                        I suppose you have overkilling textures sizes and complex shaders, i use 8 layers without problem.
                        Sure sure. Post pictures of your material setup, shader complexity, and landscape size.
                        Reading your recent posts on other threads I assume you're not going to help.

                        Comment


                          #13
                          Regardless of my sour answers in feedback, off-topic, and general, (because after 15+ years developing video games in the trenches watching the new generations ask for magic buttons is funny), the vast bulk of my replys are to help.

                          Of course compared to your maps mine looks like a pony poop (1x1 kilometres). But my maps are designed to run on all possible computers, while you want to push the limits of Ureal engine.


                          but I just wanted to point out, that in my opinion/experience that the number of layers does not seem to be the main culprit of drop framerate.


                          Click image for larger version

Name:	mao_02.JPG
Views:	1
Size:	170.6 KB
ID:	1124113

                          Click image for larger version

Name:	Shader complexity.JPG
Views:	1
Size:	455.4 KB
ID:	1124114

                          Click image for larger version

Name:	shader_01.JPG
Views:	1
Size:	229.9 KB
ID:	1124115

                          Click image for larger version

Name:	size.JPG
Views:	1
Size:	28.6 KB
ID:	1124118
                          Last edited by DomusLudus; 02-28-2017, 04:32 PM. Reason: more imagnes

                          Comment


                            #14
                            Originally posted by erWilly View Post
                            Regardless of my sour answers in feedback, off-topic, and general, (because after 15+ years developing video games in the trenches watching the new generations ask for magic buttons is funny), the vast bulk of my replys are to help.

                            Of course compared to your maps mine looks like a pony poop (1x1 kilometres). But my maps are designed to run on all possible computers, while you want to push the limits of Ureal engine.


                            but I just wanted to point out, that in my opinion/experience that the number of layers does not seem to be the main culprit of drop framerate.
                            Well that's the simplest form of making a landscape material. And as you mentioned, the landscape size is too small. When I mention "number of layers" I don't mean simply "number of layers", yes we can make 10 layers even with only 2 texture samplers, but setting up every layer with more acceptable settings for an AAA game and having a large landscape means a vast amount of red shader complexity, and that's where number of layers really counts and that's where the problem with UE4 becomes way more apparent than you are experiencing. And it really is disappointing a 2007 terrain tech (cryengine) performs worlds better than UE4 on this matter.

                            Comment


                              #15
                              OP is unquestionably right about the fact, that landscape performance issue exists. It has been brought up by most of the forum users, who worked with landscapes more or less seriously.Saying that you can use N layers without problem does not change it.

                              Luckily, it is relatively easy to cure, due to the nature of how accessible the source is. It is just that users would expect whole system to be way more optimized out of the box.


                              I'd share my experience of adapting to a specific case with minimal efforts.
                              After discovering that within a given task,
                              (an island with a bunch of different biomes, unrealistic to go below 14-16 layers and design guys totally refused to mind layer number per component restriction)
                              default system was a dead end, my approach to designing landscape shader became to blend two 16 texture atlases using 3 weightmaps.

                              Texture ID for first and second atlas was encoded in weigthmaps 1 and 2.
                              Weightmap 3 was used to paint second texture atlas over the first one.


                              At this point I had a working system that allowed having 32 layers painted elsewhere.
                              The obvious limit was that you could only blend two textures in the same place.
                              Even at this point, with all the redundant math for extracting textures out of atlas and dealing with mips, even considering it was done 3 times for first atlas, it still ran appreciably faster than if I would dare to do something like that using default layers.

                              For further improvement, atlases were replaced with arrays and branching was introduced to skip unnecessary projections for tri-plannar mapping.

                              Concluding, I'd like to say that particular approach is just one of many feasible ways to deal with landscape performance issues and restrictions, and here goes its readily obvious advantages and disadvantages as viewed against default system:

                              Pros:
                              • High number of layers
                              • Lower pixel shader cost
                              • Lower memory footrpint
                              • Framerate consistency
                              • No inflation for shader compile times



                              Cons:
                              • Only able to blend 2 textures at any given place(It is possible to expand this, but on practice it did not feel limitting at all)
                              • Relies on custom weightmaps generated in external software and/or changes to the editor painting tool
                              • Relies on alternative way to get physical surface from landscape.
                              • Efforts of implementation.





                              Lastly, I'm going to re-link two feature requests, that I think are quite relevant to landscapes and virtually everyone should casts their votes on:
                              Generate landscape physical surface from material
                              dynamic flow control support for material editor
                              Last edited by Deathrey; 02-28-2017, 06:37 PM.

                              Comment

                              Working...
                              X