I’m currently trying to improve the time it takes for Horde agents to stream switch.
We have some Horde workspaces that contain multiple streams. The delta between these streams can be large so, when an agent switches back and forth between streams, the time it takes for stream switching can be up to 20mins.
We could give each stream their own workspace to avoid the stream switching entirely but it comes with an increase in disk space usage, and thus cost.
I’m curious about how you have this set up on Fortnite where I’m assuming that you face a similar challenge.
Would you mind sharing how you approached the issue on your side?
Yes, this is an issue for Epic as well unfortunately. Horde tries to mitigate some of this with what’s called managed workspaces. This caches files which can then be re-used when switching streams. So cost should be lower than using a normal P4 workspaces. This works ok for streams that are similar. But we try to avoid this for streams that are very different, as the switching costs are high. It’s something we want to address better and I appreciate you highlighting your issue with this as well.
Are the streams similar? Say, release streams for same title or completely different titles?
It’s for the same title but the delta can still be significant.
But do you mind sharing how Fornite is set up to mitigate this issue?
Are you stream-switching on all streams, or are you giving a workspace for each stream, or maybe a mix of both based on the delta between streams (grouping similar streams in the same workspace but separating the others)?
We use split roots, just one stream per workspace where we know there would otherwise be heavy stream switching. And dedicated agents per stream. This does end up costing more in storage though so definitely comes with a trade-off. Virtual streams are also used to subset files, one example is with agents running the initial “Setup Build” step.