Announcement

Collapse
No announcement yet.

Time for some uncomfortable criticism and a sweet *** solution

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    #16
    It does get annoying that we never get a version of the engine where everything works. Something is always broken. Also, multi-monitor performance has been super poor since 4.17 and nobody cares.

    Check out my discord -> https://discord.gg/kQdVwJ3

    Follow us on twitter to get updates on new products and special offers -> https://twitter.com/BlackFangTech

    Black Fang Technologies' products -> https://www.unrealengine.com/marketp...20Technologies

    Comment


      #17
      Originally posted by TheJamsh View Post
      Fortnite shipped to iOS and Android using 4.19, albeit a likely slightly modified version - what better kind of testing is there than that?

      I find it extremely hard to believe Epic ships completely untested code, perhaps aside from minor modifications.
      I would argue that it probably isn't 4.19 "slightly modified" but instead a completely separate fork where they work on new features, pick the cool ones and jam said features into the "consumer" branch over and over again until they don't get compilation issues.

      This would explain the sheer amount of frustrating regressions that grow more then the unique bug list seems to.

      But hey, I'm sure before long we'll get the usual "Well you have source, fix it yourself" arguments in due time - which itself is a flawed argument when you take all the variables into account. I have found with the rise of their popular titles, quality control in the engine as a whole feels extremely lacking. Which I get, they need more warm bodies on Fortnite and all that - they're chasing the dollar, like any company that wants to turn a profit should do.

      However, I do feel like they're chasing the closest carrot depending on what brings in the money now in terms of what brings the money in in the long run. As I said, I have no issue with Fortnite and the resources undoubtedly being pushed to that (RIP Paragon) but I really do take issue when the biggest threat to my existing pipeline and user experience is hampered not from bugs but constant regressions of existing bugs because there was no unit testing and seemingly a pair of eyes just to check if jamming code into a separate branch has broken anything major.

      A good example of this would be the material system, which currently has a bug where it'll infinitely compile imaginary shaders if you tweak a parameter value. I mean for a bug like that to slip through seems insane to me - the material system is one of the core features of the engine and to be able to shader leak the engine in less than 10 seconds after opening it up with just a single material node to me affects my confidence in trusting Epic with my projects and client work.

      I'm not saying "Let's all boycott Unreal Engine until they give us attention" but what I am saying is (at least to me) quality in the builds we're receiving is definitely slipping down a steep, steep slope and if unit testing or some other such thing isn't integrated it'll turn into a slope where the climb back up simply isn't worth the time or money.
      KITATUS
      "Information shouldn't be behind a paywall, It should be free for all!"

      Comment


        #18
        There's tons of "unit test" classes all over the engine source code.
        Specially older more stable modules... But mostly they are all C++ functions only.
        | Savior | USQLite | FSM | Object Pool | Sound Occlusion | Property Transfer | Magic Nodes | MORE |

        Comment


          #19
          Originally posted by BrUnO XaVIeR View Post
          There's tons of "unit test" classes all over the engine source code.
          Specially older more stable modules... But mostly they are all C++ functions only.
          So a unit testing framework is there, it just needs to be used in a sane and committed way.

          Comment


            #20
            Originally posted by KITATUS View Post

            I would argue that it probably isn't 4.19 "slightly modified" but instead a completely separate fork where they work on new features, pick the cool ones and jam said features into the "consumer" branch over and over again until they don't get compilation issues.

            This would explain the sheer amount of frustrating regressions that grow more then the unique bug list seems to.

            But hey, I'm sure before long we'll get the usual "Well you have source, fix it yourself" arguments in due time - which itself is a flawed argument when you take all the variables into account. I have found with the rise of their popular titles, quality control in the engine as a whole feels extremely lacking. Which I get, they need more warm bodies on Fortnite and all that - they're chasing the dollar, like any company that wants to turn a profit should do.

            However, I do feel like they're chasing the closest carrot depending on what brings in the money now in terms of what brings the money in in the long run. As I said, I have no issue with Fortnite and the resources undoubtedly being pushed to that (RIP Paragon) but I really do take issue when the biggest threat to my existing pipeline and user experience is hampered not from bugs but constant regressions of existing bugs because there was no unit testing and seemingly a pair of eyes just to check if jamming code into a separate branch has broken anything major.

            A good example of this would be the material system, which currently has a bug where it'll infinitely compile imaginary shaders if you tweak a parameter value. I mean for a bug like that to slip through seems insane to me - the material system is one of the core features of the engine and to be able to shader leak the engine in less than 10 seconds after opening it up with just a single material node to me affects my confidence in trusting Epic with my projects and client work.

            I'm not saying "Let's all boycott Unreal Engine until they give us attention" but what I am saying is (at least to me) quality in the builds we're receiving is definitely slipping down a steep, steep slope and if unit testing or some other such thing isn't integrated it'll turn into a slope where the climb back up simply isn't worth the time or money.
            This pretty much sums up what I feel so far. If there is unit test deployed (and used), then easy bugs should not have slipped easily. TBH, they (Epic) are probably scrambling for ways to generate profit, and listening to us (and fixing all bugs) is not the high priority right now.

            Comment


              #21
              I believe this is the business model - triple A customers and investors pay for the company to run. We're pretty lucky that we get UE4 as a free, excellent framework to use but it can still be vastly improved with a change in development culture. Otherwise the natural conclusion is that we should be using a framework whose business model is our market and go to Unity instead. Which would be a **** shame but it also feels like something that's going to happen as people get fed up with the shortcomings. A complete move to Unity is being discussed in the dev group I am a part of.
              Last edited by Antidamage; 04-05-2018, 04:48 AM.

              Comment


                #22
                Originally posted by Antidamage View Post
                I believe this is the business model - triple A customers and investors pay for the company to run. We're pretty lucky that we get UE4 as a free, excellent framework to use but it can still be vastly improved with a change in development culture. Otherwise the natural conclusion is that we should be using a framework whose business model is our market and go to Unity instead. Which would be a **** shame but it also feels like something that's going to happen as people get fed up with the shortcomings.
                Yes that is also true - when I first heard about UE4 charge only *measly* $19 per month subscription, I think wow.. can they make it financially?? With all supports from UE4 experts/staffs...

                Comment


                  #23
                  Interesting stuff. I'll admit I know next to nothing about unit testing. The majority of my programming experience (~20 years) is actually non-professional, working on personal projects; which isn't to say that testing couldn't have been helpful, just that when working alone there is less pressure and more of a tendency to want to spend time on the fun functionality. I recall learning unit tests way back at uni, and finding it all boring as hell. I think the way it was taught didn't help - I recall examples where the tests were not so far away from: int x = 10; check(x == 10); and I think that sort of thing tended to put me off.

                  Anyway, with regards to UE4. I find it interesting the number of times I see people assume that Epic must be aware of this and that, or know the best way of doing things, etc, "because it's Epic". I don't think there are many in the community that dig as often and as deep into the engine code as I do. My honest opinion is that, as impressive as the resulting tech is, the engine in terms of code quality is a complete abomination. It's clear that over the years it's grown by continuously tacking on and forcing in new bits and pieces in whatever way achieved the desired result with minimum required work, short term. I won't go into specifics, but I'm convinced the core of the code is now borderline unmaintainable, and the bugs and regressions are the inevitable result. There are so many interdependencies it simply wouldn't be possible to write unit tests at this point, the code would need to be rewritten from the ground up.

                  You're right that this doesn't prevent Epic from changing their approach when it comes to new code, but then it's a question of culture, and changing the culture that's led to the current state of the codebase. Some of the more modern code is actually far better designed, but it seems to vary a lot, probably just depending on which engineer wrote it. I wouldn't hold my breath for any large scale changes.

                  You mention discussions of moving to Unity. I've had a look at Lumberyard recently in my spare time, and the design of their new architecture which they're transitioning to is flat out beautiful. Being a contractor I obviously have to work with what clients are using, but I can only hope LY gains some traction and it becomes viable for me to spend more time on that codebase and less on UE4's!

                  Comment


                    #24
                    ****. Maybe it is time to check out Lumberyard, that's the second time I've heard that. Has the documentation improved? It was abysmal last year.

                    A major reason why Unity is such a big draw is because it not only has mobile support but its mobile support is solid.

                    Comment


                      #25
                      I'm speaking purely from a programmer's perspective, I've no idea about the higher level usability. As for programming docs, last I looked into it was a couple of months back, still minimal and often out of date. So my gut feeling is, at this point it's still way, way off from being usable if you don't have a sizable team of experienced engineers; it just looks promising long term.

                      What made me want to check it out initially was less about the state of Epic's code and more about the state of their interaction and responsiveness to non licensees. Not that I expect engineers to be able to answer my every question, but generally this just dropped off a cliff right about the time they dumped the monthly subscription, and hasn't stopped falling since. It's long since I gave up even trying to submit bug reports or PRs. Probably naive to think that LY might be any better in that respect, and current evidence isn't too encouraging, but I'm willing to be optimistic...

                      Comment


                        #26
                        Originally posted by KITATUS View Post
                        I'm not saying "Let's all boycott Unreal Engine until they give us attention" but what I am saying is (at least to me) quality in the builds we're receiving is definitely slipping down a steep, steep slope and if unit testing or some other such thing isn't integrated it'll turn into a slope where the climb back up simply isn't worth the time or money.
                        I agree with everything you said, I just don't know if Unit Testing will solve any of it or will just cause more problems with even slower deployment.

                        Edit: Also Antidamage I know it's frustrating but can you avoid cursing, else I gotta do my moderator thang.
                        Last edited by TheJamsh; 04-05-2018, 08:58 AM.

                        Comment


                          #27
                          It just sounds like you just don't have any experience with unit testing or the methodologies I've been describing. There's an easy solution to that: try them out. I outlined above everything you need to know to get started reading about it.

                          Nobody who has used good development methodologies like that makes the argument that they do exactly the opposite of what they're intended to do. In this case they save time and labour. "It's too much work to fix" is only an argument used against unit testing out of ignorance.

                          Not sure where you're getting swearing from. The forum aggressively filters out a few words that aren't swear words though.

                          Comment


                            #28
                            I'm far from experienced on the subject, but I get the basic premise and I agree that in an ideal world it makes total sense to have software test itself repeatedly and automatically to ensure developers aren't introducing new problems. Work smart not hard etc.

                            So I'm not disagreeing with you at all - but the general vibe I'm getting from this thread so far is "why don't you just do this really simple thing, and it'll solve all of the problems". It's really not that simple at all, and it definitely won't stop the engine shipping with bugs no matter how extensive. If that was the case, every man and his dog would be doing it. This also assumes Epic don't already do some kind of internal unit testing - but we have no idea about their internal processes so this is all purely speculative.

                            As for me for example, it's not as easy as just "trying it out" at all, I'd have to do some pretty exhaustive reading on the subject followed by a lot of design and planning to integrate it into my daily workflow, which a lot of people simply can't afford (or just plain aren't interested in, especially if they have a workflow that works for them).

                            Just this guys opinion!

                            ---

                            As for the forum stuff, yes it filters things out - but we still see the unfiltered version, it's still against the rules and technically we're supposed to add infractions or warnings regardless. Just sayin'
                            Last edited by TheJamsh; 04-05-2018, 10:13 AM. Reason: Removed pointless quoting

                            Comment


                              #29
                              You're getting that vibe because it is that simple. I've converted several workplaces to unit testing and CI and it's not a big ask. The biggest issue is developers digging in their heels because they don't like change and don't understand the benefits, very similar to the point of view you're coming from right now.

                              Other than that you have a pretty jaded view of engine issues and that's Epic's fault for not managing the UE experience better. But they can be fixed. They get fixed all the time. It's not that much of a stretch to imagine that much of the testing can become painless and automated, and that the work to do that can be done piece by piece as part of a new development methodology. A development methodology that has far-reaching benefits from virtually every perspective. We're basically discussing seatbelts for programming.

                              There is no way Epic do internal unit testing to the degree of what I'm describing. Period. Like Bruno said there's some there, but it's for very low level functionality, probably a lot of the macro stuff. Code that is designed around unit testing looks nothing like what's in the engine. There isn't much to speculate about, you can't unit test the vast majority of the engine code as it is. Is the code bad because you can't unit test it, or is the unit testing not there because the code is bad? You don't solve a chicken/egg problem with circular logic, you just break the cycle.

                              You don't need to set up a unit testing framework and write tests to learn about it. Here's a very good presentation that discusses TDD using symfony and behat. Totally worth a look through: https://www.slideshare.net/kamiladry...hp-and-symfony

                              To be frank if you're not willing to learn about the subject then introducing a counterpoint to it is pointless and at worst harmful to what is a hugely beneficial process. So take the time to check it out and you might become a convert.

                              Edit: here's a video that covers off nicely everything I've been talking about:


                              And here's an example from PHPSpec, which is an example of the skeletonizer tool I've been describing (start from 50 seconds in):

                              The important thing to note here is the speed he can work at. It's creating all of that code for him, he's just plugging in units of work to the methods it creates based on his test functions.

                              If he was using BDD he wouldn't even have to make test functions, the BDD parser would have done that for him.
                              Last edited by Antidamage; 04-05-2018, 11:09 AM.

                              Comment

                              Working...
                              X