First off, congrats on the 4.7 release, many many fine additions. I can’t wait until I get back from vacation next week to roll up my sleeves and get all my projects upgraded to the latest version.
However, your release strategy has a few wrinkles that could stand to be ironed out. I have 30+ years of professional software development experience making reliable bug free high availability commerce software that has Billions $ worth of transactions per year used by multiple Fortune 500 companies, so please hear me out.
Once you publish the first preview of a release, say 4.8.1 there should be a code freeze put into effect on that branch. If you have a ton of stuff you want to add, just hold off on publishing .1 until you’ve fleshed it all out. It will be ok if we have to wait a bit before we see it, I promise. The only changes made on that branch after that .1 preview should be bug fixes, no new functionality and no refactoring. You could make some minor exceptions, but only for experimental areas. Once the bugs come in, you get them fixed and publish the next preview only when you actually believe it is a viable release candidate. All bug fixes should be peer code reviewed with focus on unintended side-effects and made with a minimal footprint. As in, you’re not allowed to refactor a bunch of stuff to fix a bug because you don’t understand what’s wrong with the existing code so your solution is to just rewrite it all. To make high quality software, it is critical the developers need to be working with the mindset these aren’t merely previews, they need to really be viable release candidates. It’s clear this is not currently the case now, especially considering you call them “previews” and not “release candidates” which I would highly recommend reconsidering. Only after a release candidate has been out for at least 2-3 weeks with no known reproducible crash bugs or show stopper bugs left should it be allowed to go final. Once fully vetted, then the branch should be tagged as a full .0 release build. Meanwhile during this preview process, everyone is free to go hog wild with whatever new functionality they like, as long as it’s on other branches. I think you will find once you completely eliminate the feature-creep in your release cycle, everyone’s lives will be much easier.
As a customer, I expect each subsequent preview release .1, .2, .3, etc to be successively more refined so I can see we’re smoothly progressing to something rock solid. Instead with 4.7 previews, it started off not too bad, but some of the earlier previews had blatant, immediately noticeable bugs such as iphone deploy not working, no sound, etc. These high profile regression bugs just cause more work for everyone if released, even if you do try to list them in the known issues list. UEAnswers was littered with people frustrated with these bugs, only to be told they were known issues with the release. Well if you had even waited one more day or two and fixed the immediately obvious issues, it would have saved us and your own support staff a lot of time in the long run addressing them. Not to mention I would wager if you weren’t adding new functionality, those regression bugs wouldn’t have popped up in the first place. Then in previews 7 & 8 when you were making noises that the final would be soon, suddenly things really got crazy. The BP component stuff (while indeed absolutely amazeballs and I can’t wait to redo certain parts of my projects with them) was way way too disruptive to show up this late in the preview cycle. As much as I love having them, they really should have been pushed off into 4.8. Preview 8 was out maybe a week tops, critical bugs were still coming in and yet you made it final. I can’t say for sure, but it looks like even more changes were made between preview 8 and what was actually the final .0 release. This is another thing that should not be allowed to happen, whatever was tagged as the last preview should be the 100% exact point that is tagged as the final release. A diff between 4.7.8 and 4.7.0 should have zero differences.
As your customers, we need every 4.X.0 release to be absolutely as rock solid as possible. It doesn’t do much good to upgrade from 4.5 or 4.6 to 4.7 when you’re just trading one set of critical known bugs for another new set of critical known bugs. Getting a new release with lots of new functionality is always great, but at the end of the day if the release is not reliable enough where we can actually publish & ship a game with it, it’s ultimately unusable in the real world. If you want a lot of very happy customers, don’t leave it to us to individually pull it all down from github and go through every commit to cherry pick all the fixes we need to stabilize our engine. Just ship stable releases to begin with and it’s a given you will crush your competition.
Please know I really really love this engine and I spend a lot of my not-so-free time filling out bug reports because I want this product to be the best it can be. It’s clear you have a very hard working team busting their butts making a great product. I know if you guys just focus a little more on working smarter instead of working harder, you can make final releases that are shining examples of how an engine is done right. I hope you take this constructive criticism to heart and take a new approach to the 4.8 release. Cheers.