Frequency of updates to the master branch

A while ago (half a year?) you changed the system so that you no longer merged individual commits to master and instead merged everything as huge batches of commits from development branches. I think the reason for switching to that system was to make master more stable.
Now that system in general is a bit problematic since it’s no longer possible to cherry pick individual commits and merge them into the local branch that’s used for development of the game. But master is definitely quite stable now, I don’t really see more bugs in master than there are in releases like 4.13 or 4.14. So I’m fine with that.

But what I really think should be improved is the frequency of merges from the development branches into master.

Let’s say someone uses master from today (7th december). Looking back from today, the last dates where other branches were merged:

Dev-Rendering: ----- Dec 03, 05 days ago
Dev-Core: ----------- Nov 23, 14 days ago
Dev-Framework: ---- Nov 16, 21 days ago
Dev-Blueprints: ----- Nov 11, 26 days ago
Dev-Editor: ---------- Oct 19, 49 days ago
Dev-Mobile: ---------- Oct 08, 60 days ago
Dev-Networking: ---- Oct 04, 64 days ago
Dev-Physics: --------- Oct 03, 65 days ago
Dev-VR: ------------- Sept 30, 68 days ago

On average thats 41 days.

Since I’m working on VR, I’m mostly interested in dev-rendering and dev-vr. The last dev-rendering merge war 5 days ago, that’s good. Before that it was another 3.5 weeks though. But the last dev-vr merge was whole 68 days ago! When you think about what a company the size of Epic can produce in 68 days that’s a lot. You can’t really call your master branch “master” any more like this, I think. It’s not master, it’s some branch where code needs on average almost 1.5 months to get into.

This get’s especially noticeable when into a branch like 4.14 that’s prepared for the 4.14.1 hotfix now there is merged some really interesting new stuff that’s not yet in master, because master has to wait for the huge commits from the development branches, while 4.14 gets some important commits directly. One quick example:

9 days ago, the 4.14 branch got this really interesting commit: https://github.com/EpicGames/UnrealEngine/commit/af9d6262473492e4e9850133a128c95663ebb5be

So, 1 ms more performance for VR! That’s super awesome! But those that use master to get exactly something like that as fast as possible don’t have that yet. It’s not in master, so it will probably get merged with the next merge from Dev-VR where the last one was 68 days ago. That’s absurd when you think about what the reason for using master is, to get everything as fast as possible without having to wait for stable releases.

So I guess you could say that currently master is broken. Are there any plans from Epic to fix this? :frowning:

I think that was more than 1.5 years ago. More or less since the Kite release. 2015

Mainly looks like Epic is focus in their games, can probably count with the fingers of one hand how many devs in Epic are working only in the engine.

Near all merges are from changes in Paragon or other of their games.
And yeah I would like to go back to the old system, and get more changes and fixes in the engine but… well, since they are with the games idk if that is possible…
Is hard to spot the changes or fixes, and usually I see how some of the changes, break old fixes etc, in the second part other times you can’t even spot changes cause the change is so big and half of the files don’t even load.

So yeah I liked the old system more.

I wouldn’t even say I liked the old system more. I see the advantages in stability the new system brings, and stability is also worth a lot.

But with the new system, merges from development streams should happen roughly once a week, not more than 68 days apart.

All I would like to contribute to this would be #SavePaper2D

Bit more programming frequency of updates to Paper2D would be much appreciated. Thank you.

Hi John.

Thanks for the feedback. This is an area that we want to drive improvement as well, and we hope to start on it early next year (2017).

I’m glad to say this is not true. You’d need the hands of many people. :slight_smile:

Thanks Stephen!

Very nice to see that you’re already aware of this being a problem :slight_smile:

While I’m interested in seeing more frequent merges, I’m most interested in seeing individual commits instead of the giant monolithic merges. This allows us to merge only parts we want, which isn’t really practical right now.

Also, is there any possibility of being able to get more feedback on long running PRs? As an example, I’ve been maintaining a PR, texture arrays, which was originally submitted over a year ago. I definitely get that these aren’t quick things to evaluate and accept/decline and in this specific case I’ve had some limited information from Olano, but I would put more time into this if it for sure had a future, but for now I’m left wondering.

I concur, current giant monolithic merges are simply useless, to the point where sometimes it’s even impossible to see their full content due to their size, let alone pick any single contained commit. Could we at least get various dev streams exposed on GitHub, so that we can look at individual changelists?

If it’s so complicated to mirror everything to git, why not just allow read access to the perforce repo?

Hey [MENTION=9]Stephen Ellis[/MENTION], are there any updates about this? :slight_smile: We really need the ability to cherry pick individual commits.

+1.
I actually like master to stay the way it is now, but to have all te seprate branches pushed to github. (At those that do not have any NDAs).

Thats probably because not all branches can be make public (like Odin was for Switch which was NDA).

Don’t see the problem there. Just don’t allow access to those specific branches then until the NDAs expire.

I’m glad to say that my previous statement holds true, and we’ve been working on this recently. Stay tuned for an update once all testing is complete.

amazing

Can’t find on github.

Huh? It’s the latest commit on master: https://github.com/EpicGames/UnrealEngine/commit/1547db70c95bba3c9bccde4cbd20dc11c297a033

Hah right. I saw it, just never made it to the bottom of the page ;o.

Hey Stephen, any update on it? I would think it doesn’t take more than two weeks if its actively being worked on, unless you are doing some awesome stuff no one expects :cool:

The bulk of the engineering work is done and I’ve seen it functional in a private repo. We are currently testing and reviewing workflows to ensure we are able to maintain confidentiality where required. This stage may take some time, and we look forward to sharing the results as soon as possible.

Cheers

Awesome, thanks! :slight_smile: