Announcement

Collapse
No announcement yet.

UMG and Slate - New Clipping System in 4.17

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • UMG and Slate - New Clipping System in 4.17

    For 4.17, we're upgrading Slate and UMG's clipping system to fix some long standing problems with clipping artifacts! The new system does change previous behavior and some things had to be deprecated so I'm posting this here prior to the docs being prepared for 4.17+, so that anyone using the preview builds or building from main will know the deal.

    Slate Clipping 1.0

    The Basics
    • Axis aligned clipping - only functioned in layout space.
    • Every widget clipped
    • All clipping done in the pixel shader (required 6 floats per vertex)
    • Allowed batching across clipping rects, since the rects were part of the batch data.
    • Render transforms were ‘special’ in that they allowed rendering outside the clipping rect provided by the parent - with side effects when the layout rect got clipped.


    The system Slate has had from the beginning was based entirely around axis aligned boxes for clipping. When we introduced UMG and soon after added render transforms circa 4.5 the clipping system was marginally adjusted to clip in a special rotated layout space. It was for all intents and purposes still axis aligned, it just render transformed the final axis aligned clipping rect for rendering so that render transformed elements still clipped themselves correctly after applying the render transform.

    However - this leads to a very obvious but difficult to solve problem with existing usages. None of the other parent clipping rects merged with the render transformed clipping rect, and so it lead to several problems that were noticeable at the edges generally.

    https://issues.unrealengine.com/issue/UE-4659

    Here's a common example of a problem with the current clipping system in Slate,

    Click image for larger version

Name:	image9.png
Views:	1
Size:	22.0 KB
ID:	1224196 -> Click image for larger version

Name:	image2.png
Views:	1
Size:	34.5 KB
ID:	1224197


    Slate Clipping 2.0

    The Basics
    • A clipping zone is an arbitrary quad.
      • Axis Aligned Clipping Zones use Scissor Rects to clip (avoids unnecessary pixel shader ops).
      • More complex clipping zones use the stencil buffer to compose an arbitrary stack of clipping zones into the stencil buffer which is then used to clip draw calls.
    • By default very few widgets clip now, mostly just the scroll panels and editable text fields.
      • Render Transforms are no longer ‘special’, any transform can allow you to render outside of another widget, provided it doesn’t clip.
    • Slate can longer batch across clipping zones. This free’d up 6 floats in the Slate vertex format, and we no longer need to clip in the pixel shader.


    Now that clipping occurs in render space of widgets, we no longer need to concerned with weird clipping issues like on the left, instead we can choose to enable clipping on the outermost widget and now any transform we apply to inner widgets is correctly clipped at the edges,

    Here's an example of a button that *was* stretched across the corner of a canvas and rotated with the intent being to clip it at the edges, in Clipping 1.0, this would have clipped the un-rotated version.

    Clipping 1.0 Clipping 2.0
    Click image for larger version

Name:	image7.png
Views:	1
Size:	7.9 KB
ID:	1224198 Click image for larger version

Name:	image8.png
Views:	1
Size:	7.4 KB
ID:	1224199
    Normally you don’t really need to worry about changing the clipping state of a widget. Consider your game UI, if there were a button that was clipping the text inside of it, that is probably a bug. If the player is unable to read the button text, that’s a problem. Mostly you lean on clipping in scroll panels, or in cases where you can’t control the length of the text and need to clip it.

    Here’s an example of what you might see happening and the choices you can make.

    Left - No Clipping
    Center - Clipping is enabled on the Text
    Right - Clipping is enabled on the button

    Click image for larger version

Name:	image5.png
Views:	1
Size:	16.5 KB
ID:	1224200

    Notice the difference it makes who is responsible for clipping, since we clip to bounds things like padding do not factor in, if the button is set to clip (Right), the contents will run all the way to the edge before they clip, whereas if the text is responsible (Center), then the text clips based on things like button content padding that reduce the space the text has to fit in.

    In addition to the simple axis aligned examples of clipping, the system supports arbitrary clipping quads, stacked on top of each other. Below is an example of a rotated clipping zone with a rotated scroll box that also clips, properly clipping its children.

    Click image for larger version

Name:	image4.png
Views:	1
Size:	143.8 KB
ID:	1224201

    While the above example seems rather silly, being able to combine multiple arbitrary clipping zones together is a prerequisite to being able to introduce 3D transforms and projections, since any widget being clipped with a projection, would need perform clipping in projected space.

    Conversion Guide

    Enabling Clipping

    To ‘turn on’ clipping, you’ll adjust the Clipping property of the widget, e.g.

    Code:
    SNew( SBorder )
    .Clipping(EWidgetClipping::ClipToBounds)
    [
            ...
    ]
    For UMG, all UWidgets also now have a Clipping property.

    99% of the time, you’ll only need to concern yourself with the two most important Clipping states, EWidgetClipping::Inherit and EWidgetClipping::ClipToBounds.

    EWidgetClipping::Inherit - This widget does not clip, it obeys whatever clipping / culling it was passed in from a parent widget.

    EWidgetClipping::ClipToBounds - This widget clips content to the bounds of the widget.

    There are 3 other states available for clipping (but they're advanced)

    EWidgetClipping::OnDemand - This widget clips to its bounds when it's Desired Size is larger than the allocated geometry the widget is given. If that occurs, it work like EWidgetClipping::ClipToBound. We actually default STextBlock to OnDemand in the Slate, but keep it as Inherit in UMG. This was done primarily because text is so often clipped in the editor due to resized docking panels and the like that it made sense.

    NOTE: This mode was primarily added for Text, which is often placed into containers that eventually are resized to not be able to support the length of the text. So rather than needing to tag every container that could contain text with [Yes], which would result in almost no batching, this mode was added to dynamically adjust the clipping if needed. The
    reason not every panel is set to OnDemand, is because not every panel returns a Desired Size that matches what it plans to render at.

    EWidgetClipping::ClipToBoundsWithoutIntersecting - This widget clips to its bounds. It does NOT intersect with any existing clipping geometry, it pushes a new clipping state. Effectively allowing it to render outside the bounds of hierarchy that does clip.
    NOTE: This will NOT allow you ignore the clipping zone that is set to [EWidgetClipping::ClipToBoundsAlways].

    EWidgetClipping::ClipToBoundsAlways - This widget clips to its bounds. It intersects those bounds with any previous clipping area.
    NOTE: This clipping area can NOT be ignored, it will always clip children. Useful for hard barriers in the UI where you never want animations or other effects to break this region.

    Deprecated APIs

    All of the existing FSlateDrawElement::Make____(...) calls have been deprecated. We no longer pass a clipping rect into every draw call, so just remove the clipping rect from the function call, and you’ll be using the new version.

    If you were doing something magic and special with the clipping rect, consider if even need to now. Since not every widget clips, you may be able to simply ignore what you were doing if the job of this code was to maybe trick the clipping system to allow you to move the clipping zone.

    The SScissorRectBox is dead. It no longer is necessary, since every axis aligned clipping rect is a scissor rect, please delete it at your earliest convenience. You can simply replace the job it was doing by making its most direct child widget have clipping enabled.

    Custom Clipping

    Suppose you need to clip internally in your widget, for example the SProgressBar needs to clip the progress bar drawing at arbitrary locations based on progress. In order to add your own clipping, you’ll do the following in OnPaint,

    Code:
    OutDrawElements.PushClip(FSlateClippingZone(AllottedGeometry));
    //TODO Do your drawing here, or child widget paint calls.
    OutDrawElements.PopClip();
    The FSlateClippingZone is the arbitrary clipping area in window space, it can be initialized using several methods that should allow fairly easy conversion from existing code.

    If you also need your custom clipping to affect Hit Testing, you will need to also push your clipping zone onto the Hit Test Grid,

    Code:
    OutDrawElements.PushClip(FSlateClippingZone(AllottedGeometry));
    Args.GetGrid().PushClip(FSlateClippingZone(GetCachedGeometry()));
    //TODO Do your drawing here, or child widget paint calls.
    OutDrawElements.PopClip();
    Args.GetGrid().PopClip();
    Note that the clipping rect for the Hit Test Grid is using the cached geometry, instead of the AllottedGeometry. In OnPaint AllotedGeometry is in Window Space. The Hit Test grid is not in window space, it’s in Desktop Space, so you have to use geometry that you’d get in Tick.
    Wrapped Clipping States

    Sometimes a widget that has a clipping state isn’t actually responsible for clipping. For example, the SScrollBox lets you publicly change the clipping state like any other widget, but when the SScrollBox was constructed it set bClippingProxy = true, what that does is makes it so that when Slate goes to render that widget, it ignores the clipping state of that widget.

    Internally the SScrollBox informs another nested widget that it needs to clip, or do whatever clipping it was told to perform or not perform. Additionally when users change the clipping state, the SScrollBox overrides virtual void SWidget::OnClippingChanged() so that it knows when to mirror the new state to the nested private widgets.

    Culling Changes

    First, all the places where you were taking in MyClippingRect in your OnPaint call, rename it to MyCullingRect.

    Now that clipping is performed in render space, we still perform culling using a simple bounding box (though it is based on the bounding box of the render transformed clipping zone and layout version), additionally clipping/culling might become more nuanced over time, so if you have a custom panel and were doing, MyClippingRect.IntersectionWith in order to cull widgets that can’t possibly be drawn, you should use SWidget::IsChildWidgetCulled instead e.g.,

    Code:
    for (int32 ChildIndex = 0; ChildIndex < ArrangedChildren.Num(); ++ChildIndex)
    {
            FArrangedWidget& CurWidget = ArrangedChildren[ChildIndex];
            if (!IsChildWidgetCulled(MyCullingRect, CurWidget))
            {
                // Paint this widget.
            }
    }
    Something to keep in mind with the culling approach used in Slate (and this hasn’t changed), is that if the parent widget is culled (the blue rect below), the yellow child circle widget, would be culled even if it has a transform that allowed it to render entirely outside the bounds of blue, even if it used EWidgetClipping::ClipToBoundsWithoutIntersecting, as that flag is only inspected by the direct parent.

    Click image for larger version

Name:	image3.png
Views:	1
Size:	6.0 KB
ID:	1224202

    Debugging Clipping

    The Widget Reflector has been updated to now show the clipping state for widgets.

    Click image for larger version

Name:	image1.png
Views:	1
Size:	178.8 KB
ID:	1224203

    There are several new console commands to assist with debugging clipping and culling.

    Slate.ShowClipping 1 - This command shows yellow outlines for all Axis Aligned clipping rects (Scissor Rects), and red outlines for all stencil clipping quads.

    Click image for larger version

Name:	image6.png
Views:	1
Size:	63.3 KB
ID:	1224204

    Slate.DebugCulling 1 - Allows you to better understand how the culling in a panel is working...or not working correctly. It disables clipping on the GPU, but everything continues as normal, so it allows you to see all the things rendering outside the bounds of clip zones and see if they are being culled when you expect them to be.

    Slate.EnableDrawEvents 1 - Enables draw events so that when you’re debugging with RenderDoc or something similar that it’s easier to understand the batches or clipping state transitions. This is enabled by default in Debug builds.


    Misc Notes

    Here's some notes I've made as I've upgraded and fix some fallout from this change internally,

    1) Roll out panels - e.g. animating the Desired Size Scale on Border widgets, to have a panel slide out. You'll want to enable clipping on the content of that border the desired size scale won't have any impact. (I'm thinking about just having SBorder clip if people use DesiredSize scale, in the same way I do implied clipping of the Scale Box with options like Fit X/Y...etc.

    2) Some things that were previously invisible, may actually be visible now. Strangely enough, some things that should have always been visible, now that visibility correctly takes into the render transform, and layout transform, instead of just one of them, may cause you to suddenly see widgets that use to get culled because their layout position was moved off screen (or outside the container widget), but if it *should* have been visible, it now will be.
    Last edited by Nick Darnell; 06-26-2017, 02:18 PM.

  • #2
    Looks good, Nick.

    Comment


    • #3
      Fantastic! I'm looking forward to improving our UI with this.

      Comment


      • #4
        4.17 is gunna be awesome! :-)

        Thanks for sharing!
        =========================

        Protonwar - Your body is ready!

        Comment


        • #5
          Awesome improvement, thanks for the hard work!
          ARTSTATION: https://www.artstation.com/artist/noisestorm

          Comment


          • #6
            Really excited about this, must have been a massive effort

            Comment


            • #7
              Woow this is awesome Nick.
              UE4 Development [TARTARUS | Object Inspection System]
              Let's Connect [Twitter]

              Comment


              • #8
                Thanks Nick,

                I know this has been on the TODO list for awhile! Keep up the awesome work.
                Abatron Development Thread
                www.abatrongame.com
                Steam Store

                Comment


                • #9
                  This is awesome Nick, thanks for the detailed post!

                  Out of interest, some of my games have a heavy UI. Are there any additional performance concerns with the new clipping system?

                  Comment


                  • #10
                    This is really awesome! Can't wait to test it!

                    Comment


                    • #11
                      Originally posted by TheJamsh View Post
                      Out of interest, some of my games have a heavy UI. Are there any additional performance concerns with the new clipping system?
                      Shouldn't be, all told it should be a wash, unless you needed to do a lot of clipping for some reason - then it would affect batching. Some computation on the CPU got easier, some harder. GPU load should be less, now that there are fewer interpolants, and a smaller vertex buffer, and less pixels that could be drawn are, but at the same time, clipping isn't much needed for most games.

                      Comment


                      • #12
                        Nice work. Some of my UI still uses the old "Canvas"- system. Can the clipping be applied there somehow?

                        Comment


                        • #13
                          Originally posted by Braindrain85 View Post
                          Nice work. Some of my UI still uses the old "Canvas"- system. Can the clipping be applied there somehow?
                          No - the FCanvas system doesn't share any code with Slate.

                          Comment


                          • #14
                            Originally posted by Nick Darnell View Post
                            No - the FCanvas system doesn't share any code with Slate.
                            Nick, can you please look at this Slate question I posted a short time ago (as well as the linked question)?

                            Comment


                            • #15
                              Great work! Can't wait til 4.17 is final.

                              I have a few questions on how the new clipping features work using a SceneCapture2D UMG widget.

                              1) Does stencil clipping work on a SceneCapture2D widget? If it does, will it also clip the RenderTarget drawn into by the SceneCapture2DComponent, so we are not process pixels not shown? Example: To use in a Voronoi dynamic split-screen https://www.youtube.com/watch?v=YFRWvxYJ9hc

                              2) In the custom clipping OnPaint(), does the geometry need to be are rectangle or axis-aligned rectangle (4 vertices)? Could it be a triangle or arbitrary (5, 6, 7, ... vertices) ?

                              Background: I want to create something similar to the dynamic split-screen in the video above, but would like to clip the RenderTarget drawn into by my SceneCapture2DComponent. When the split-screen feature is active (ie. the players are far apart), the method in the video renders the entire screen for each view, even though only half is shown. I looking for suitable methods in UE4 I can use in creating a custom object/component.

                              Thanks

                              Comment

                              Working...
                              X