P4 Sparse stream support on Horde

Development Pipeline & Plugins
, ,
1

Hello, I’m basing this question on top already existing Q for "[UnrealGameSync support for sparse [Content removed]

We found Sparse stream as interesting feature that could save us valuable resources during creating “temporary streams” - especially because on current UE architecture we’re running into big costs on creating new streams while copying all files -> possible P4 outages. Unfortunatelly apart from UI tools like UGS not having support, also Horde is having issues fetching files from those, so we cannot use them even for like “sparserelease” streams.

(I’m attaching screenshot of what we got when tested)

In recent talks with Helix, we’ve received info that they are going to be optimizing and improving quality of Sparse streams in next releases of p4d (P4 Server).

Docs: Sparse streams | P4 CLI Documentation (2025.1)

Are there any plans for supporting Sparse stream on Horde and if yes what is the ETA for that?

Thanks,

Jan

2

Hi Jan,

Thanks for the report. I’m not too surprised this fails right now with Horde. The default sync model is called ManagedWorkspace, which does things a bit different from a normal p4 sync, as you may have noticed. This to optimize stream switching we have internally at Epic.

But many or most people using Horde likely aren’t interested in that and would prefer a vanilla Perforce syncing model. We’ve also seen a use-case for this internally on machines that do not switch streams, for example agents doing incremental builds.

There’s now a new materializer called “PerforceMaterializer” which simply uses P4 in the most standard way possible. The code is in and we are currently running some tests with it, so a bit experimental at the moment (conforms not behaving). But we are going to stabilize this and ship it, probably for 5.8. I also gave it a quick try with sparse streams and seems to work fine.

I see you mention 5.7 as your version. Are you perhaps already running latest Horde or building from source? If so you can do this today with:

"workspaceTypes": {
    ...
    "Incremental": {
        "base": "Win64Base",
        "identifier": "Foo+Main+Inc",
        "stream": "//Foo/Main-Incremental",
        "incremental": true,
        "method": "name=perforce" // <-- Line of interest
    },
    ...
}

With a disclaimer that there will be some sharp edges