Bugs: Back Logged vs Fixing

Hi Epic!

Could you outline how you determine which bugs to work on and which bugs to backlog? Is there some high-level policy in place? I’ve not done an extensive search of the forums, so apologies if this has been addressed. We can provide feedback here and the Epic team can determine if suggestions can be incorporated into the policy / procedure… Thanks!

teak

I did some digging and found this brief outline of the proceedings.

tumblr_mmt1mtuoNF1rl52wjo1_400.gif

Backlogged Bugs are actually never fixed.
And Won’t fix Bugs are never fixed, too. (What a plot twist)

#MakeBugsGreatAgain

Backlogged bugs do get fixed… eventually. There is a term on issues called WON’T FIX. Those we’re doomed on… What would be helpful is we know the thought process of the determination… Will clear the air!

teak

<laughs> Thats constructive…

A coin flip would be so much easier.

There’s very little in the way of policy. It’s up to each team lead / system owner what they feel should be backlogged. Basically - it all comes down to, what do you think you will reasonably get to and what do you think is a priority. If it’s not - then backlog it. We have finite resources and can’t fix everything. So things like crashes we can reproduce, regressions, problems with new systems are examples of things we try to fix ASAP. Problems with older systems, that have been there forever that have known workarounds, or would just be really really hard to fix right now without a lot more resources, tend to get backlogged until an opportune time presents itself to tackle that issue.

Which for me is where I take into account votes for items, e.g. Unreal Engine Issues and Bug Tracker (UE-4659), was backlogged as of about a week ago. It had been backlogged for probably about 1.5 years Why? Well - we knew that the render transform stuff was going to have limitations with the clipping system in Slate, and we knew that fixing that meant completely redoing Slates clipping system and changing the default behavior. This would of course break all existing customized uses of Slates clipping system, so big nasty hairy problem. There were more important things to tackle at the time with UMG and Slate, so that issue was backlogged.

I removed it from the backlog about a week ago, because I finally had a some time to work on redoing Slates clipping system, and had some good ideas on how to do it. Still, as expected it’s a really difficult problem - I have over 200 files checked out right now to make this change,.

We simply can’t fix everything, so it comes down a judgement call. Additionally if we don’t touch a bug within 6 months, a script automatically backlogs it, unless someone touches the bug, and resets the clock.

Thanks a ton Nick for your post! Aren’t you supposed to be enjoying your weekend? Take care…

teak

Let me be productive:

Why not look at the type of games that selling the most units and use that as a barometer to judge what issues should be fixed or not. Ark: Survival Evolved has sold over 5 million copies. That means Epic has potentially earned over $5 million worth of revenue. PlayerUnknown’s Battlegrounds has sold 1.7 million copies earning Epic over $2.5 million worth revenue. Why not look into bug fixes and performance increases for these type of games which are actively making Epic tons of money (literally 7 tons).

An improved vegetation system and virtual texturing would also improve the quality of these as they work their way through Early Access. Spline Decals would allow them to create additional roads and paths for their games and their sequels with significantly reduced overhead. In turn these games get a better reputation, they sell more copies, and Epic makes millions more. Moreover, when studios release their own sequels to Ark, PU: Battlegrounds’, etc., Epic will make even more cash from games like these.

Moreover, updating these systems and fixing bugs that affect games like these would allow us to create new open world games without any of the performance issues plauging Ark and PU: Battlegrounds. This in turn would allow our own UE4 open world games to put a much better foot forward with better performing games, our reputations intact, and in turn make more money for Epic as there is clearly a market for similar games. It’s true that you can’t fix everything. But it’s also true that you can make reasonable decisions to prioritize features and bugfixes for the type of games that are bringing in the most revenue for Epic.

I think the lesson is to have good faith on Epic staff… there are lots of reasons of why things are not as they may seem…

I am sorry but you just gave examples of 2 games where devs either did no optimization runs at all or use horrible methods (I mean… PUB didn’t used Instanced Foliage for trees until 1 patch ago…). And their bad performance is not a result of bugs.
You can’t just say “EPIC PLOX FIX MY GAME”…

Also I bet 7 million is not a lot of money for Epic. It’s not nothing. But it’s not a lot.
I think what people are asking mostly (and me) is for Epic to start being transparent about bugs they decide to fix and ones they decide to skip.
And they should never let bugs lay for 2-3 years, especially critical bugs like traces not registering hits on landscape or vehicles that are broken as f**k, shadows making big objects look like flat shaded.

Not everybody is making open world games in UE4… and people who argue like you make Epic focus on shinny features to please community instead of real issues.

I would agree… They do a great job. But, additional transparency into their throught process on bug classification will help in setting expectations for the community…

teak

You bring up great points, but the purpose of this thread is to shed light on how Epic classifies bugs…regardless of who may or may not be affected.

Nick already responded here with how they do things and that’s awesome! Would like something more concreate, but…that’s me. Not sure if you can make a formal policy on this, but it would be nice to have one created and published.

teak

If that $7 million dollars isn’t “a real issue”, they’re free to give that money back to the developers. Everyone may not be making open world games but open world games are making Epic money. As such they should definitely factor into how Epic prioritizes the bug fixes.

And that statement is based on what? You have no data to support that. Again crappy argument that brings nothing constructive to the discussion.

Sales figures of games released using Unreal Engine 4, the prices of those games, and the 5% revenue that Epic takes. Which other games using UE4 have sold as much as Ark? PU: B is the number 1 selling game on Steam for the past month and is the second highest streamed game on Twitch right now. Those numbers are taken straight from the Steam API and Epic’s website.

There’s also been false positives by staff marking bugs as being fixed only for the issues to persist. This also needs to be addressed. We also need to have a conversation about how comfortable the community should be working with a person who repeatedly displays an incapability of understanding bugs presented to him, in spite the of proof attached. Is there any accountability for such staff?

Looking at issues,

Highest voted “Fixed” bug: 51 votes.
**Highest voted “Backlogged” bug: 78 Votes.
**
Highest voted “Fixed” (Added) feature request: 62 votes.
**Highest voted “Backlogged” feature request: 74 votes.
**
If we go on like that, Backlogged bugs and feature requests have usually double times more votes on them. It doesn’t seem logical to say every single time community needed something it’s been a nasty hairy issue for Epic to care. Seems more like, Community needed it but Epic didn’t so it was backlogged because we only dedicate more resources to fixing nasty hairy issues when it benefits Epic Games.

Yes. Like saying you care about community while you don’t.
They’ve made millions of dollars from only a couple indie UE4 titles yet those games suffer tremendous amount of negative reviews because Epic isn’t adding the proper tools for them and fixing critical bugs in return of the millions of dollars they’re getting from them. Spending a fraction of that profit on actually addressing community’s issues could make everyone happy instead of blaming lack of resources all the time…

Don’t get me wrong I agree with you on bugs and how this matter is handled. I am not happy about it because many of those bugs impact my professional and hobby project.
I just think you should not bring arguments like “I checked on SteamDB and PUB sold 1.7mill units! EPIC must be rich!”. Because you have no clue about money Epic makes from licenses sold to AAA studios, money from their own titles and other revenue streams we have no clue about.

Let’s see how this topic evolves when everybody is back im the office and they have a chance ( :smiley: ) to respond.

EDIT: Well Nick already did. So let’s just see how this evolves in the future :slight_smile:

The issue isn’t that Epic is rich. It’s how revenue generated from UE4 is invested back into the engine, specifically whether such revenue prioritizes how bugfixes are determined. If not, should they be? The perspective I’m taking is whether Epic should nurture a developer ecosystem that pays back into itself by focusing more, not exclusively, on the needs of those developers that bring in revenue for Epic. This ecosystem in turn would benefit all of us because, as we make similar games as those other developers, we can bring in more revenue for Epic by releasing more stable products.

If you disgaree, I respect your opinion (much love and respect). But to me it seems that progress on UE4 has recently been scatterbrained and this could help bring the company a bit more focus.

Actually we do.

If they have $15 million dollars to save land, they have enough money to solve a few engine bugs.

I am sorry but how Tim spending his OWN money has anything to do with Epic Games? (for me he can spend all the money he has on private land, good for him)
You do know Epic Games is a company driven by a board, and investors to respond to?

EDIT: I will just stop responding, because topic about bugs turned into salt-fest with argument of money which brings no solution to the original issue.