Announcement

Collapse
No announcement yet.

The Quest to combine Texture Atlasing and Landscape painting.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • replied
    Originally posted by iniside View Post
    Just as a side note. Did you tried to use streaming levels ? I mean cut landscape into chunks, reduce visibility to be quite low, mask everything else with fog. and stream landscae chunk as player get closer to them ?

    Or you said you had problems with collision mainly. Then I wonder if it would be possible to create three stage streaming. If player is not close enough to streamed level only, something like:
    1. Level is not loaded at all because player can't see it.
    2. Player can make vague details of next streaming chunk, so we stream simplified version, without any collision data, nav mesh etc.
    3. Player is close so we stream everything.
    Just wonder if you tried it. I honestly never done big landscape, but these are vew ideas that poped into my head.
    Would work but did not try that. If I have to cover the whole area with very dense fog we are back to 2004. Need a solution without having to do that.

    Streaming in different distant levels could work but is likely going to be a really annoying thing to set up. Having to ensure what streams and unstreams when, and without getting obvious transitions.

    Thanks for the suggestions though.

    Leave a comment:


  • replied
    This has been a very informative conversation and that method is much more effective thank you for sharing it. I shall explore the "BlendAngleCorrectedNormals" function and see how this impacts performance

    Leave a comment:


  • replied
    Originally posted by Hourences View Post
    Ah if it is a 2k height map that is not too bad. Neither is 400 mb memory. If you consider that most gaming pcs and next gen consoles have 8 or so GB in memory, you can take 400 for a terrain. According to my tests earlier it was really the collision data that had a hit on memory and was the biggest concern.

    Solus was meant to be large and open, but I had to scale down because of the difficulties in making such big worlds... That is why I now got islands, means allow me to have small areas surrounded by emptiness. I am not saying it is impossible to do large open worlds, but it is definitely not the strongest point of the engine, and the overhead and risk it brings along is too big imo to be worth it.

    Lod wise foliage is special. I have not tried this, but I assume the foliage will either never lod, or it will lod in groups. Foliage is instanced, it batch renders parts together. So it can't lod individual objects within a batch. It will likely lod the entire group at once or none of it.
    And those groups would have to be pretty big, to lower the total number of groups that exist in the level. A max draw distance would have to be set also. And even then... Just grasses alone already is going to result in probably thousands if not tens of thousands of clusters on such a big scale...

    Streaming levels/world browser may help, but standing on a mountain and overlooking the whole area would still result in the above problems.

    It very quickly all gets out of hand.
    Just as a side note. Did you tried to use streaming levels ? I mean cut landscape into chunks, reduce visibility to be quite low, mask everything else with fog. and stream landscae chunk as player get closer to them ?

    Or you said you had problems with collision mainly. Then I wonder if it would be possible to create three stage streaming. If player is not close enough to streamed level only, something like:
    1. Level is not loaded at all because player can't see it.
    2. Player can make vague details of next streaming chunk, so we stream simplified version, without any collision data, nav mesh etc.
    3. Player is close so we stream everything.
    Just wonder if you tried it. I honestly never done big landscape, but these are vew ideas that poped into my head.

    Leave a comment:


  • replied
    Ok I see what you are doing now; the texturesample nodes didn't paste with the other ones so I wasn't sure of the context.

    Screen will do weird things there, and you aren't blending the blue channel. For example if you screen 1,1,0 and -1,1,0 you will get a result of 1,1,0 which isn't really a meaningful blend between those two normals. You can debug this using constant3vectors and the "DebugFloat3Values" material node.

    What I usually do in most cases is add both normals together, multiply by 1,1,0.5 and then normalize. If you consider one of them to be macro or dominant you could use the "BlendAngleCorrectedNormals" function but it will cost a few more instructions.

    Leave a comment:


  • replied
    Thanks for checking it out.

    I've been using the blend_screen to combine the red channels of two different normals , I found that if I used append it takes the second channel as a green one which it is not. I tried adding the normalize onto the output of the first append pictured below, the results were not desirable.

    Click image for larger version

Name:	CombiningNormals.jpg
Views:	1
Size:	432.8 KB
ID:	1051132
    Last edited by MrNexy; 06-07-2014, 05:54 PM.

    Leave a comment:


  • replied
    If I had to guess it would be because the broken up texture data was not imported as a normal map but as a regular texture and still needs to be biasscaled back into the -1 to 1 range. Also make sure to uncheck the sRGB flag in the texture properties. For 0-1 data, use constantbiasscale node set with bias -.5 scale 2.

    I am not sure exactly what screen is doing for you in this case, why not use append for all the channels? Give me a bit I will fire up the editor and see the material maybe I will see whats going on.

    *edit* ok I checked the nodes you sent. did you normalize the result? When you modifiedy the R,G normals which would alter them from the specified blue channel, you will need to renormalize. Also I am still not 100% sure the role the screen blend is playing here. If the normals are in the -1 to 1 range, it could stomp some sign information the way that works. It inverts and multiplies and then inverts again, which is like adding but won't go beyond 1.0. What specifically are you using that for?
    Last edited by RyanB; 06-07-2014, 05:27 PM.

    Leave a comment:


  • replied
    I hope thats not a sign that I'm doing anything wrong
    I do have a quick question for you though. I have been using blend_screen to merge the red channels of two different normals and subsequently the green channels of those respective normals to make a new normal. After that I append a blue channel from the original normal, I've noticed that the new normal doesn't smoothly deform the terrain as well as its separate components did. Do you have any tips that could remedy this ? I've included the node below for reference.

    Thanks for taking the time to contribute to my thread

    Code:
    Begin Object Class=MaterialGraphNode_Comment Name="MaterialGraphNode_Comment_84"
       Begin Object Class=MaterialExpressionComment Name="MaterialExpressionComment_62"
       End Object
       Begin Object Name="MaterialExpressionComment_62"
          SizeX=512
          SizeY=272
          Text="Dirt_1+Sand_1"
          MaterialExpressionEditorX=-144
          MaterialExpressionEditorY=6480
          Material=PreviewMaterial'/Engine/Transient.PreviewMaterial_10'
       End Object
       MaterialExpressionComment=MaterialExpressionComment'MaterialExpressionComment_62'
       NodePosX=-144
       NodePosY=6480
       NodeWidth=512
       NodeHeight=272
       NodeComment="Dirt_1+Sand_1"
       NodeGuid=96E1E915472D6BA91C35EA8E9E5261AD
    End Object
    Begin Object Class=MaterialGraphNode Name="MaterialGraphNode_1284"
       Begin Object Class=EdGraphPin Name="EdGraphPin_7734"
       End Object
       Begin Object Class=EdGraphPin Name="EdGraphPin_7733"
       End Object
       Begin Object Class=EdGraphPin Name="EdGraphPin_7732"
       End Object
       Begin Object Class=MaterialExpressionMaterialFunctionCall Name="MaterialExpressionMaterialFunctionCall_618"
       End Object
       Begin Object Name="EdGraphPin_7734"
          PinName="Result"
          Direction=EGPD_Output
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1286.EdGraphPin_7739"'
       End Object
       Begin Object Name="EdGraphPin_7733"
          PinName="Base (V3)"
          PinType=(PinCategory="required")
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1288.EdGraphPin_7746"'
       End Object
       Begin Object Name="EdGraphPin_7732"
          PinName="Blend (V3)"
          PinType=(PinCategory="required")
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1282.EdGraphPin_7722"'
       End Object
       Begin Object Name="MaterialExpressionMaterialFunctionCall_618"
          MaterialFunction=MaterialFunction'/Engine/Functions/Engine_MaterialFunctions03/Blends/Blend_Screen.Blend_Screen'
          FunctionInputs(0)=(ExpressionInputId=A591DEF24B8DE9173A27AE80DCEF3259,Input=(Expression=MaterialExpressionTextureSample'/Engine/Transient.PreviewMaterial_10:MaterialExpressionTextureSample_32',OutputIndex=1,InputName="Blend",Mask=1,MaskR=1))
          FunctionInputs(1)=(ExpressionInputId=A2AC794848506B955B2643B4D7D5E64E,Input=(Expression=MaterialExpressionTextureSample'/Engine/Transient.PreviewMaterial_10:MaterialExpressionTextureSample_34',OutputIndex=1,InputName="Base",Mask=1,MaskR=1))
          FunctionOutputs(0)=(ExpressionOutputId=085286B14E9452E3918853BB8CB1AB7D,Output=(OutputName="Result"))
          MaterialExpressionEditorX=-96
          MaterialExpressionEditorY=6528
          Material=PreviewMaterial'/Engine/Transient.PreviewMaterial_10'
          Outputs(0)=(OutputName="Result")
       End Object
       MaterialExpression=MaterialExpressionMaterialFunctionCall'MaterialExpressionMaterialFunctionCall_618'
       Pins(0)=EdGraphPin'EdGraphPin_7732'
       Pins(1)=EdGraphPin'EdGraphPin_7733'
       Pins(2)=EdGraphPin'EdGraphPin_7734'
       NodePosX=-96
       NodePosY=6528
       NodeGuid=F21E47E4474B6BB5FBDF169C06E43608
    End Object
    Begin Object Class=MaterialGraphNode Name="MaterialGraphNode_1285"
       Begin Object Class=EdGraphPin Name="EdGraphPin_7737"
       End Object
       Begin Object Class=EdGraphPin Name="EdGraphPin_7736"
       End Object
       Begin Object Class=EdGraphPin Name="EdGraphPin_7735"
       End Object
       Begin Object Class=MaterialExpressionMaterialFunctionCall Name="MaterialExpressionMaterialFunctionCall_619"
       End Object
       Begin Object Name="EdGraphPin_7737"
          PinName="Result"
          Direction=EGPD_Output
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1286.EdGraphPin_7738"'
       End Object
       Begin Object Name="EdGraphPin_7736"
          PinName="Base (V3)"
          PinType=(PinCategory="required")
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1288.EdGraphPin_7747"'
       End Object
       Begin Object Name="EdGraphPin_7735"
          PinName="Blend (V3)"
          PinType=(PinCategory="required")
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1282.EdGraphPin_7723"'
       End Object
       Begin Object Name="MaterialExpressionMaterialFunctionCall_619"
          MaterialFunction=MaterialFunction'/Engine/Functions/Engine_MaterialFunctions03/Blends/Blend_Screen.Blend_Screen'
          FunctionInputs(0)=(ExpressionInputId=A591DEF24B8DE9173A27AE80DCEF3259,Input=(Expression=MaterialExpressionTextureSample'/Engine/Transient.PreviewMaterial_10:MaterialExpressionTextureSample_32',OutputIndex=2,InputName="Blend",Mask=1,MaskG=1))
          FunctionInputs(1)=(ExpressionInputId=A2AC794848506B955B2643B4D7D5E64E,Input=(Expression=MaterialExpressionTextureSample'/Engine/Transient.PreviewMaterial_10:MaterialExpressionTextureSample_34',OutputIndex=2,InputName="Base",Mask=1,MaskG=1))
          FunctionOutputs(0)=(ExpressionOutputId=085286B14E9452E3918853BB8CB1AB7D,Output=(OutputName="Result"))
          MaterialExpressionEditorX=-96
          MaterialExpressionEditorY=6624
          Material=PreviewMaterial'/Engine/Transient.PreviewMaterial_10'
          Outputs(0)=(OutputName="Result")
       End Object
       MaterialExpression=MaterialExpressionMaterialFunctionCall'MaterialExpressionMaterialFunctionCall_619'
       Pins(0)=EdGraphPin'EdGraphPin_7735'
       Pins(1)=EdGraphPin'EdGraphPin_7736'
       Pins(2)=EdGraphPin'EdGraphPin_7737'
       NodePosX=-96
       NodePosY=6624
       NodeGuid=48432B3F42D66AC67D82AA95069047E1
    End Object
    Begin Object Class=MaterialGraphNode Name="MaterialGraphNode_1286"
       Begin Object Class=EdGraphPin Name="EdGraphPin_7740"
       End Object
       Begin Object Class=EdGraphPin Name="EdGraphPin_7739"
       End Object
       Begin Object Class=EdGraphPin Name="EdGraphPin_7738"
       End Object
       Begin Object Class=MaterialExpressionAppendVector Name="MaterialExpressionAppendVector_56"
       End Object
       Begin Object Name="EdGraphPin_7740"
          PinName="Output"
          PinFriendlyName=" "
          Direction=EGPD_Output
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1287.EdGraphPin_7741"'
       End Object
       Begin Object Name="EdGraphPin_7739"
          PinName="B"
          PinType=(PinCategory="required")
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1284.EdGraphPin_7734"'
       End Object
       Begin Object Name="EdGraphPin_7738"
          PinName="A"
          PinType=(PinCategory="required")
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1285.EdGraphPin_7737"'
       End Object
       Begin Object Name="MaterialExpressionAppendVector_56"
          A=(Expression=MaterialExpressionMaterialFunctionCall'"MaterialGraphNode_1285.MaterialExpressionMaterialFunctionCall_619"')
          B=(Expression=MaterialExpressionMaterialFunctionCall'"MaterialGraphNode_1284.MaterialExpressionMaterialFunctionCall_618"')
          MaterialExpressionEditorX=112
          MaterialExpressionEditorY=6544
          Material=PreviewMaterial'/Engine/Transient.PreviewMaterial_10'
       End Object
       MaterialExpression=MaterialExpressionAppendVector'MaterialExpressionAppendVector_56'
       Pins(0)=EdGraphPin'EdGraphPin_7738'
       Pins(1)=EdGraphPin'EdGraphPin_7739'
       Pins(2)=EdGraphPin'EdGraphPin_7740'
       NodePosX=112
       NodePosY=6544
       NodeGuid=0F8BE18540CD3A2A83D066A95E947ECD
    End Object
    Begin Object Class=MaterialGraphNode Name="MaterialGraphNode_1287"
       Begin Object Class=EdGraphPin Name="EdGraphPin_7743"
       End Object
       Begin Object Class=EdGraphPin Name="EdGraphPin_7742"
       End Object
       Begin Object Class=EdGraphPin Name="EdGraphPin_7741"
       End Object
       Begin Object Class=MaterialExpressionAppendVector Name="MaterialExpressionAppendVector_57"
       End Object
       Begin Object Name="EdGraphPin_7743"
          PinName="Output"
          PinFriendlyName=" "
          Direction=EGPD_Output
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1227.EdGraphPin_7466"'
          LinkedTo(1)=EdGraphPin'"MaterialGraphNode_1227.EdGraphPin_7486"'
          LinkedTo(2)=EdGraphPin'"MaterialGraphNode_1227.EdGraphPin_7506"'
          LinkedTo(3)=EdGraphPin'"MaterialGraphNode_1227.EdGraphPin_7526"'
          LinkedTo(4)=EdGraphPin'"MaterialGraphNode_1227.EdGraphPin_7546"'
       End Object
       Begin Object Name="EdGraphPin_7742"
          PinName="B"
          PinType=(PinCategory="required")
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1283.EdGraphPin_7730"'
       End Object
       Begin Object Name="EdGraphPin_7741"
          PinName="A"
          PinType=(PinCategory="required")
          LinkedTo(0)=EdGraphPin'"MaterialGraphNode_1286.EdGraphPin_7740"'
       End Object
       Begin Object Name="MaterialExpressionAppendVector_57"
          A=(Expression=MaterialExpressionAppendVector'"MaterialGraphNode_1286.MaterialExpressionAppendVector_56"')
          B=(Expression=MaterialExpressionTextureSample'/Engine/Transient.PreviewMaterial_10:MaterialExpressionTextureSample_33',OutputIndex=3,Mask=1,MaskB=1)
          MaterialExpressionEditorX=224
          MaterialExpressionEditorY=6544
          Material=PreviewMaterial'/Engine/Transient.PreviewMaterial_10'
       End Object
       MaterialExpression=MaterialExpressionAppendVector'MaterialExpressionAppendVector_57'
       Pins(0)=EdGraphPin'EdGraphPin_7741'
       Pins(1)=EdGraphPin'EdGraphPin_7742'
       Pins(2)=EdGraphPin'EdGraphPin_7743'
       NodePosX=224
       NodePosY=6544
       NodeGuid=A6B0544943BC9E56680599A1498675C2
    End Object

    Leave a comment:


  • replied
    That instruction count seems pretty low to me. Anything under 100 is viable on mobile actually although sometimes we have to go further and check fullyrough and disable the directional lightmaps. That usually seems to cut another 20-30% although if not on mobile there is usually no reason to go so far.

    You don't need to add the instruction counts. It's just 99 if you have lightmaps but only 75 if you have only dynamic lighting.

    Leave a comment:


  • replied
    Actually I meant the number of instructions. The memory matters but is of lesser concern than the number of instructions the material does. At the top of the material editor viewport, it should say something like "250 instructions with dynamic lighting" and then another line listing a bigger number for static light maps.
    That is very handy to know, thank you.
    These are the stats for the material, they add up to 207 instructions thus far (at first glance).

    Code:
    Info Texture samplers: 6/16
    Info Base pass shader with static lighting: 99 instructions
    Info Base pass shader with only dynamic lighting: 75 instructions
    Info Vertex shader: 33 instructions
    Last edited by MrNexy; 06-07-2014, 04:24 PM.

    Leave a comment:


  • replied
    Originally posted by MrNexy View Post

    At the moment I'm recompiling the shaders after some changes ,there are 20/50 or so layers compiled so far and the pixel shader memory is around 9 MB engine. I'll get some more layers painted after compilation and see how high it goes. What is rough is the navigation memory hehehe.
    Actually I meant the number of instructions. The memory matters but is of lesser concern than the number of instructions the material does. At the top of the material editor viewport, it should say something like "250 instructions with dynamic lighting" and then another line listing a bigger number for static light maps.
    Last edited by RyanB; 06-07-2014, 02:42 PM.

    Leave a comment:


  • replied
    No append vector is almost a free operation so it is faster. For derive normal z, the shader must solve the z value for each pixel using the Pythagorean theorum. It saves texture samples but adds gpu rendering cost. Only necessary if you are hitting some kind of texture sample limit. It doesn't really save memory either since two dxt1 textures are the same as a dxt5.

    What I forgot to mention was how you would be packing two normals into a single dxt5. Red and green are normal 1, blue and alpha are normal 2.
    Last edited by RyanB; 06-07-2014, 02:22 PM.

    Leave a comment:


  • replied
    Hey there RyanB

    Along the same lines, you can pack two normal maps into one Dxt5 texture by breaking each into just rg and using the node derivenormalZ inside of the material to regenerate the B channel.
    I'll get that integrated into my current shader, its probably more effective than what I've been using which is the Append Vector node.

    I am curious what the pixel shader cost is. At some point these 'memory savings tricks' can become expensive.
    At the moment I'm recompiling the shaders after some changes ,there are 20/50 or so layers compiled so far and the pixel shader memory is around 9 MB engine. I'll get some more layers painted after compilation and see how high it goes. What is rough is the navigation memory hehehe.

    Leave a comment:


  • replied
    That is pretty crazy. We did something similar for Sol but not quite so much of it. Along the same lines, you can pack two normal maps into one Dxt5 texture by breaking each into just rg and using the node derivenormalZ inside of the material to regenerate the B channel.

    I am curious what the pixel shader cost is. At some point these 'memory savings tricks' can become expensive.

    Leave a comment:


  • replied
    Current Landscape Material Update

    Hey there folks .

    Just wanted drop by with an update on my current material setup.
    I've gone with the texture packing route and by packing 4 gray scale textures into the R,G,B and Alpha channels.For the normals I had to take a more straight forward approach by simply using one texture per normal. I would have used the same packing for the normals but their quality suffers from it, I think it may have something to do with the way textures are handled during compression.

    In addition to the 4 original textures I was able to create 6 new textures and normals by recombining the original textures and their normals. With 5 color variations I ended up with a palette of 50 colors.





    The shader is a mess of strings but modular and not too difficult to make adjustments to one step at a time. The shader compilation count and time can be long (around 250,000 in my case) but once everything is compiled and you're not making any changes to the shader you're good to go I haven't noticed any performance hits or increase in memory usage for textures or shaders even with all of the different layers in use.

    Let me know your thoughts ^^

    -MrNexy

    Leave a comment:


  • replied
    Ah if it is a 2k height map that is not too bad. Neither is 400 mb memory. If you consider that most gaming pcs and next gen consoles have 8 or so GB in memory, you can take 400 for a terrain. According to my tests earlier it was really the collision data that had a hit on memory and was the biggest concern.
    Thats good to know My collision data (navigation data ? ) is around 200 , this is definitely an area of concern moving forward.

    Solus was meant to be large and open, but I had to scale down because of the difficulties in making such big worlds... That is why I now got islands, means allow me to have small areas surrounded by emptiness. I am not saying it is impossible to do large open worlds, but it is definitely not the strongest point of the engine, and the overhead and risk it brings along is too big imo to be worth it.
    You're absolutely right! I've thought about scaling things back on numerous occasions and especially when I've run into technical issues on the UE4 side. Its still early days I suppose and there are plenty of improvements that are likely to only be implemented in versions leading up to 4.4.

    Lod wise foliage is special. I have not tried this, but I assume the foliage will either never lod, or it will lod in groups. Foliage is instanced, it batch renders parts together. So it can't lod individual objects within a batch. It will likely lod the entire group at once or none of it.
    And those groups would have to be pretty big, to lower the total number of groups that exist in the level. A max draw distance would have to be set also. And even then... Just grasses alone already is going to result in probably thousands if not tens of thousands of clusters on such a big scale...

    Streaming levels/world browser may help, but standing on a mountain and overlooking the whole area would still result in the above problems.

    It very quickly all gets out of hand.
    You're right about the foliage LOD as far as I can tell. Thankfully I won't be creating vast grasslands or tall mountainous peaks, granted that does mean that I will have to be extra careful. Things like cluster count,size and draw distance are certainly a concern. I've been pondering how to best use streaming levels in open outdoor spaces, subterranean applications of level streaming work very well as you demonstrated in your video walkthrough of Solus.

    I'm interested to see where they take things with the world browser and how it will impact creating large open worlds in a more manageable way.

    Leave a comment:

Working...
X