UE4 Preview Builds and Release Strategies

General Feedback & Requests
1

First off, congrats on the 4.7 release, many many fine additions. I can’t wait until I get back from vacation next week to roll up my sleeves and get all my projects upgraded to the latest version.

However, your release strategy has a few wrinkles that could stand to be ironed out. I have 30+ years of professional software development experience making reliable bug free high availability commerce software that has Billions $ worth of transactions per year used by multiple Fortune 500 companies, so please hear me out.

Once you publish the first preview of a release, say 4.8.1 there should be a code freeze put into effect on that branch. If you have a ton of stuff you want to add, just hold off on publishing .1 until you’ve fleshed it all out. It will be ok if we have to wait a bit before we see it, I promise. The only changes made on that branch after that .1 preview should be bug fixes, no new functionality and no refactoring. You could make some minor exceptions, but only for experimental areas. Once the bugs come in, you get them fixed and publish the next preview only when you actually believe it is a viable release candidate. All bug fixes should be peer code reviewed with focus on unintended side-effects and made with a minimal footprint. As in, you’re not allowed to refactor a bunch of stuff to fix a bug because you don’t understand what’s wrong with the existing code so your solution is to just rewrite it all. To make high quality software, it is critical the developers need to be working with the mindset these aren’t merely previews, they need to really be viable release candidates. It’s clear this is not currently the case now, especially considering you call them “previews” and not “release candidates” which I would highly recommend reconsidering. Only after a release candidate has been out for at least 2-3 weeks with no known reproducible crash bugs or show stopper bugs left should it be allowed to go final. Once fully vetted, then the branch should be tagged as a full .0 release build. Meanwhile during this preview process, everyone is free to go hog wild with whatever new functionality they like, as long as it’s on other branches. I think you will find once you completely eliminate the feature-creep in your release cycle, everyone’s lives will be much easier.

As a customer, I expect each subsequent preview release .1, .2, .3, etc to be successively more refined so I can see we’re smoothly progressing to something rock solid. Instead with 4.7 previews, it started off not too bad, but some of the earlier previews had blatant, immediately noticeable bugs such as iphone deploy not working, no sound, etc. These high profile regression bugs just cause more work for everyone if released, even if you do try to list them in the known issues list. UEAnswers was littered with people frustrated with these bugs, only to be told they were known issues with the release. Well if you had even waited one more day or two and fixed the immediately obvious issues, it would have saved us and your own support staff a lot of time in the long run addressing them. Not to mention I would wager if you weren’t adding new functionality, those regression bugs wouldn’t have popped up in the first place. Then in previews 7 & 8 when you were making noises that the final would be soon, suddenly things really got crazy. The BP component stuff (while indeed absolutely amazeballs and I can’t wait to redo certain parts of my projects with them) was way way too disruptive to show up this late in the preview cycle. As much as I love having them, they really should have been pushed off into 4.8. Preview 8 was out maybe a week tops, critical bugs were still coming in and yet you made it final. I can’t say for sure, but it looks like even more changes were made between preview 8 and what was actually the final .0 release. This is another thing that should not be allowed to happen, whatever was tagged as the last preview should be the 100% exact point that is tagged as the final release. A diff between 4.7.8 and 4.7.0 should have zero differences.

As your customers, we need every 4.X.0 release to be absolutely as rock solid as possible. It doesn’t do much good to upgrade from 4.5 or 4.6 to 4.7 when you’re just trading one set of critical known bugs for another new set of critical known bugs. Getting a new release with lots of new functionality is always great, but at the end of the day if the release is not reliable enough where we can actually publish & ship a game with it, it’s ultimately unusable in the real world. If you want a lot of very happy customers, don’t leave it to us to individually pull it all down from github and go through every commit to cherry pick all the fixes we need to stabilize our engine. Just ship stable releases to begin with and it’s a given you will crush your competition.

Please know I really really love this engine and I spend a lot of my not-so-free time filling out bug reports because I want this product to be the best it can be. It’s clear you have a very hard working team busting their butts making a great product. I know if you guys just focus a little more on working smarter instead of working harder, you can make final releases that are shining examples of how an engine is done right. I hope you take this constructive criticism to heart and take a new approach to the 4.8 release. Cheers.

2

I believe our current process works how you’ve described but we might have some confusion around terminology. We released 4.7.0 as our first stable this last week and released the 4.7.1 hotfix to address a few critical issues found in that release. No code changes are being made to the 4.7 branch at this point unless it is for a critical bug that we intend to address in the next hotfix. All active feature development is (and has been for a couple weeks now) on the 4.8 branch, which is ‘main’ in our depot configuration. Once development for 4.8 gets to a point where we’re ready to start building a new official release we’ll branch the build from main, start the preview process once again and focus on stabilizing that branch until we get to an official release.

Preview releases are intended to allow us to work with the community to identify issues that help us build a better stable release. We are always expanding our internal QA efforts but we also recognize that there a large number of developers who are using the engine for a diversity of projects that will potentially expose issues that we might otherwise miss. The preview release process so far has helped us tremendously in capturing a large number of these issues, or in some cases augment our internal QA process by helping us generate new reproductions steps for particularly challenging bugs. We have some ideas on how to further improve the process and I’d appreciate more feedback on how we can make this more transparent and useful to developers. Thanks!

3

I kind a understand where you are coming from and Previews are definitely a nice addition to test what is coming up. But at the same time, I wouldn’t expect a subsequent Preview to be stabler as the previous, because I always considered them a special branch of allowing the community using the binary version to work with the developers directly, since that was only possible in a direct manner on the Source Code Branches.

I would expect that Epic tries to clarify a bit clearer and more to people what Preview means and even put up a red button in the Launcher. I was a lot on Answerhub during the time and a lot of people broke even Live Projects with no backups, completely not understandable since it was saying Preview, but than, different people with different backgrounds understand things a bit differently of course.

Allowing Epic to bring forth Previews to the community and those who want to engage, to collaborate to make a better Engine and take sometimes risky steps, for the diversity of use cases, is a huge thing and I suspect, no I am pretty sure that with a stable release, Epic wouldn’t release anything which is known to them breaking something.

Secondly, this being a very rolling release approach, next of using an open community approach, with handling two different streams (source and binary, generalizing here of course) this is more than I could ever want to be feel to be part of helping and creating a better Engine with those in charge.

4

I guess the misunderstanding happened somewhere between Preview 2 and Preview 7 where you announced some new features? :wink:

5

Yes, fair point on us shifting plans a bit with the 4.7 release about halfway through. One of our longer term goals, as we build up our automation processes, is to provided nightly binary builds which would arguably be a more effective method than our current release process. We’ve got quite a bit of work on our release and automation processes however, so I imagine that will still be a ways out. In the meantime as I said before we’ll continue to iterate on the current preview process and improve clarity in the launcher to hopefully avoid any further confusion or frustration.

6

Dear Epic,

So far 4.7 is going quite well for me, but I just had to quote Furroy above to honor Furroy’s impassioned and important commentary.

(I reformatted Furroy’s quote for emphasis)

I really am wanting to see a super stable build get released.

I’ve been waiting for a build that felt as stable to me as 4.3 did, a build I could release on, and so far 4.7.1 is looking really good!

I’ll be honest, I would have stayed on 4.3 if not for all the C++ compile time improvements and UObject Subsystem improvements that have been made since, it was just such a stable build and had everything I needed (I had written my own C++ UI system).

I do really love UMG though, and all of the amazing c++ improvements that have been made since 4.3, I just want a build that was as stable as 4.3 :slight_smile:


**So Many Great Ideas, A Little Too Fast**

You all have so much talent and so many great ideas, but they are getting integrated into the engine a little too fast I think, choosing "progressiveness" over "stability" a little too often.

The Multi Pre-Release Model / “Release Candidate” Model

Epic, I really like this new work flow of many many pre-release builds before doing the full release!

I’d love it even more if you could integrate what I am quoting Furroy about.

A super-stable build would be a real anchor point for those of us who are far into project development, even if it means that build has less new features.

I really need core engine features to be working at top performance at this point, not vast quantities of new stuff.


**Alternating Build Suggestion**

Maybe you could alternate "stable" and "progressive" builds?


**One build** = Lots of new stuff!!! (current mentality)

**Next build **= Stablize everything! Synergize all newly added features!


I can tell how much you want to make UE4 the best engine in the world with new stuff.

But what about making UE4 the best because it is also extremely stable?

**Maybe this alternating build idea could be of use to you!**

You just have to restrain yourselves on the build you know is the "stable" build and focus on making all the newly added parts of the engine work in the most streamlined and bug free way possible!

Then you can let loose on the "progressive" build and use this multi-pre-release model to get community input.

Happy Feedback

The C++ and Animation multi-threading improvements in 4.7 are a must-have, thanks for 4.7 Epic!

My frame rates went up by about 20-30 just by upgrading to 4.7!

I very much appreciate all your efforts and can tell just how much work and passion was put into 4.7 by all of you!

[FONT=Comic Sans MS]:heart:

Rama

7

Sorry, I wrote my original post before I left on holiday, so I’m just now getting time again to respond.

It’s not enough to just kinda sorta follow a release strategy, you have to be draconian about it. If you want to keep releasing these ‘preview’ builds to the public that are totally unstable, then I guess that is your prerogative, but perhaps you should call them ‘experimental’ builds since that’s a better term you’re already using. I would seriously argue against if it I had a voice on the team. Any version someone can pull down from the launcher comes from Epic and the quality/stability of that build reflects on Epic. The customers that want to work hand-in-hand with Epic developers aren’t using the launcher, they are pulling straight from Github. The vast majority who do use the launcher, just want to make their own games and only want stable binary releases.

All that said, the fact that you did a hot fix the very next day after 4.7.0 was released proves that it was still released too early. You had to have known those issues were there. When you had 4.7.0 ready, you could have released that to everyone and said “hey here’s a build to test out for us” aka “beta” Then your customers who don’t mind being a guinea pig (like me!) could all help stabilize it before it was considered ‘final’. And that’s when you tell the entire customer basr the new version is ready for prime time and everyone can upgrade.

Yes everyone loves new features, but stability cannot continue to take a back seat to them every single release. Stability of a final release version should always be the #1 priority.

8

Personally, I kinda liked how it is… We can download the more stable releases if you want, or you can download ‘preview’ releases when they are avail, which just like they are called, are preview releases to give you an idea of whats to come(and help Epic with bug fixes so YOU get to help and let them know your bugs, to help make the stable release, well more stable) You don’t have to download the preview ones :wink:

Either way, keep rockin’ Epic, you guys are great, love the new stuff(I do tend to work from either promoted or master branch though)