[Feature Request] Smooth pointer-driven UI (drag & drop): client-side pointer position for UMG, or client interpolation of Verse Fields

General Feedback & Requests (UEFN)
, , , , , , ,
1

Select a topic

Verse API

Tell us your feedback. What is working, what is not, and what would you change?

Summary

Pointer-driven UI (dragging an item, a slider handle, a map pin, a card) can’t be made smooth in UEFN today. Every position update has to go through Verse on the server, so a dragged widget moves in steps at the server tick rate (~30 Hz). Meanwhile the client renders at 60–144+ fps and Fortnite’s own cursor moves smoothly next to the dragged widget. I measured this and tried every route I could find. Details are below, along with some small, client-only hooks that would solve it without running creator code on the client.

What I measured (UEFN 43.00, Launch Session on PC, keyboard & mouse)

Setup: a Verse UI modal (canvas + texture_block). TouchMapping / PointerSelect give the pointer, and each move resizes two invisible spacers to place the image (no widgets are re-created). On release the server prints timing stats taken with GetSimulationElapsedTime(), plus a Sleep(0.0) loop that counts server ticks.

Metric Result (7 drags)
PointerSelect move events reaching the server ~74 per second (72–90)
Server ticks during the same drags 29.94 per second
Max gap between move events 33.4 ms = exactly one tick
Gaps over 50 ms 0, apart from one 66.8 ms hitch (one skipped tick)

So input isn’t the bottleneck. 2–3 pointer events arrive per tick, and the widget can only change once per tick, and then only after the replication round trip back to the client. I coalesced the updates to one per tick. That removes redundant work, but it can’t go above 30 Hz.

What I tried

  • Verse UI: no translation or render-transform API. Positioning is limited to canvas slots (fixed once added), padding (fixed at construction) or SetDesiredSize on spacers, all of it server-driven.
  • UMG + Verse Fields, binding a float to RenderTransform.Translation.X: the binding is created, but the widget fails to compile: “Property Transform has getter accessor. Accessor not supported on FStructProperty since it would create a temporary structure…”
  • UMG + the allowed MakeTransform conversion: the signature is (Translation: FVector2D, Scale, Shear, Angle: float). Verse Fields have no vector2 type, so a float can only feed Angle.
  • UMG SetRenderTranslation: it takes an FVector2D, and none of the allowed conversion functions build one from two floats.
  • Custom cursor: the UMG Cursor property only offers stock EMouseCursor shapes. “Custom” needs project UI settings, which UEFN doesn’t expose. So the dragged item can’t become the (OS-smooth) cursor either.

All of these are still server-driven at the tick rate, even if they worked.

Why it matters

Drag & drop is the expected interaction for inventories (including the new Custom Inventory & Items UI), crafting grids, loadout screens, card and puzzle games, map pins and sliders. On PC it’s what players expect from any inventory. Today creators have to fall back to click-to-pick-up / click-to-place.

Requests (smallest first, all presentation-only on the client, with gameplay authority staying on the server)

  1. Client-side “follow pointer” for UMG. For example, a bindable client-side source for the virtual pointer position, or a widget/slot option like “Follow Virtual Pointer” with an offset. Verse would only toggle it (start or stop the drag, choose the image) and receive the drop as an event.
  2. Opt-in client interpolation for float Verse Fields, so a server-driven value tweens between updates on the client instead of jumping.
  3. A vector2 Verse Field type, or an allowed MakeVector2D conversion, so SetRenderTranslation / RenderTransform can be bound. This alone wouldn’t be smooth, but it would at least make render-translation positioning possible.
  4. A per-widget custom cursor texture, so a dragged item can be shown as the cursor while dragging.

Related input request:

  1. Pointer move events without holding a button. Today PointerSelect only publishes events after the left mouse button is pressed, and then one per movement until release. There’s no way to subscribe to plain pointer movement (hover). An event or input action for pointer position while no button is held would enable hover tooltips, highlighting the slot under the cursor, and previews that follow the cursor before a click.

Minimal repro (Verse)

# Opens a fullscreen canvas with a square that follows PointerSelect.
# Server prints how many move events arrived vs how many ticks passed.
using { /Fortnite.com/Devices }
using { /UnrealEngine.com/Temporary/SpatialMath }
using { /UnrealEngine.com/Temporary/UI }
using { /Verse.org/Colors }
using { /Verse.org/Input }
using { /Verse.org/Simulation }
using { /Verse.org/SpatialMath }

pointer_rate_device := class(creative_device):
    var Moves:int = 0
    var Ticks:int = 0

    OnBegin<override>()<suspends>:void =
        for (P : GetPlayspace().GetPlayers(), PlayerUI := GetPlayerUI[P], In := GetPlayerInput[P]):
            PlayerUI.AddWidget(canvas{}, player_ui_slot{InputMode := ui_input_mode.All})
            In.AddInputMapping(UI.TouchMapping)
            In.GetInputEvents(UI.PointerSelect).TriggerActivationEvent.Subscribe(OnMove)
        loop:
            Sleep(0.0)
            set Ticks += 1
            if (Mod[Ticks, 90] = 0):
                Print("3s: {Moves} pointer moves, 90 ticks")
                set Moves = 0

    OnMove(Arg:tuple(player, (/Verse.org/SpatialMath:)vector3)):void = set Moves += 1

Hold the left mouse button and move: you get ~220 moves per 90 ticks. Any widget position derived from them updates at most 90 times in that window.

Thanks. Happy to provide the full device or test builds if useful.

1 Like