[TUTORIAL] Goals and Necesities AI (Story Generation Section One)


After the recent thread, I decided to start on my series of AI tutorials. I will be putting all YouTube videos for this section within this thread.
Feel free to ask questions and comment (I’d prefer if people commented on this thread instead of on YouTube)

The posts are below since I apparently have hit the max number of links I can paste


Gotta record it and edit

Subscribed so I can watch after I finish my current free-time project. In fact I’m adding it to my personal Trello so I don’t forget. :slight_smile:

Yay! Thank you :smiley:

I’m starting to realize how much content this is going to be now haha. 10 sections, each probably 4 videos long, then another section of bonus videos. jeez. Its going to be very fun :slight_smile:

Part Two going up now

Episode 1:

Episode 2: WireZappGames Tutorial: Procedural Story Generation: Goals & Necessities (P2, Episode 2) - YouTube
For some reason I can’t edit more videos into the top so this will have to do.

That’s odd, I wonder if there’s an edit limit.

Next episode is going through edits this week end. It would have already been up by now but everyone is getting sick lol

Slight update. WireZapp is no longer going to be using Unreal Engine Editor (A few reasons: A lack of runtime physics baking; Too much overhead in general; Poor shader compiling optimization; The fact of how far you need to go out of your way to make procedural geometry. All of which making the editor entirely useless to us) for any part of development. So all parts of the tutorial within unreal engine will be strictly code. We will be making our own editor though, and it may be in some tutorials down the line. Everything seen in the editor can be done with the sdk that I am making tutorials about. it just takes a little longer and is a bit more tedious. We won’t be releasing source on our editor.

In other news, here is the next part of the tutorial!

That’s a shame. Wasn’t worth trying to add it to the existing system?

We did try to do that, but the current limitation that we had in UE4 around our AI editor was that it would have taken too much time to re-implement all the graphing UI pieces that we needed for debugging the AI. Our data mining back end on our personal editor would have created too much overhead with UE4.

We did implement a little of it in UE4 to experiment if it’d be possible, but everyone in the team kept on getting clock timeout errors, and continuous blue screens of death. Its just too computationally heavy to add the debugging layer, seeing how the functionality layer uses two cores on its own.

The editor that we’re working on won’t be open source at all, but we may release it for free noncommercial usage, and TBD price/revenue percentage for commercial usage. No one following the tutorial series would want to use it as of now anyway though. Too many bugs. We only started it last week. It’ll probably be a month or so until its good enough to use in house.

I can’t give details on what exactly the debugger reports on besides the fact that it was very much needed to fine tune our data back propagation system (What we’re calling our AI LOD system). We may do a video on it eventually, just a 5 minute video or so showing it off

All very interesting, good to see you putting so much emphasis on stuff like this. Whizz-bang graphics only impress me so much.

Thanks! That’s sort of our reason for doing this in UE4. We already had our own in-house graphics engine (With full Radiosity support, billboard reflections, even physically based rendering to a degree, etc.) It looked just a tad worse than UE3.9. But in all honesty we didn’t want to take the time to update it to use more modern stuff like multithreaded rendering, we didn’t want to take the time to move to a more real-time Radiosity solution (We were using a combination between Local PRT and Instant VPL. Those bake times were painful). Most importantly, the fact that everytime we got a new programmer we had to train them to use our engine. It was chaos, none of our systems were “Normal.”

We eventually decided that it’d be better to scrap our engine, and move to a more standardized system that could manage our entire graphics side of things. We were hopping that we could use UE4 for more than just graphics, but to be honest it turned out to be easier just to start removing code and re-implementing things our way than learning their systems.

After removing the entire AI engine, we were questioning how to approach the problem at hand. We needed something that was immensely more optimized for high amounts of agents. Something that could allow us to manage close to 70,000 NPCs (We aren’t using near that amount. I think Fero has about 6.9k right now). So we came up with our back propagation engine. Only thing is, it didn’t work with finite state agents at all. So we implemented a fuzzy logic engine. But then the AI had issues inferring things that just come intuitively to you and I. Instead of moving back to finite states, and having to get rid of our LOD system, we implemented a machine learning engine.

At this point and time, we have re-purposed all of our 4 graphics programmers onto AI, as well as our Electrical Engineer onto Data mining (We even have a writer doing design lol). Right now we have so much AI programming to do, that adding a graphics engine on top of that might cause some sort of revolt in our studio. A not overworked programmer is a happy programmer :slight_smile:

Its a good thing WireZapp started as an AI firm :3

Sounds intense, keep it up! That really popular “Adventures of more DLC” game could use the competition! :slight_smile:

We have a bit of a tendency to use too many NPCs though incase you find that number appalling (Our system actually goes faster the more NPCs we have, which is why we have excessive amounts).

Like, if an agent goes farming, then sells his product to a concession stand, there are 5 AIs at work here.

  1. The agent doing the farming
  2. The agent maintaining the Entity’s inference abilities, so it knows how much money it’ll end up getting, how it’ll spend that, how it’ll store some. Etc
  3. The agent that takes the food from the farmer
  4. The agent that then takes the food from the merchant and decides how to distribute it among storefronts for later reselling
  5. The agent that manages #4’s inference abilities, which in turn allows the AI to decide how that food is distributed among the citizens, when the peaks of demand will most likely be, and to adjust the price accordingly

Obviously if you start breaking it down even more, there are more entities than that (ie: The consumer, perhaps the agent’s employer, perhaps his family that he has a “goal” to protect). Realizing how a chain of events can be represented as a series of agents instead of perhaps just two or three is one the most enabling features of our solution. It allows us to exponentially increase our simulation complexity while actually having a performance increase

We max out at 150,000 agents Queue updates per second. Since a queue can take an entity 5 - 10 minutes to complete (and are done entirely A-synchronously) in a best case scenario, our system could potentially (if we had unlimited HDD space or RAM) manage close to 90 million agents. Of course, we don’t have unlimited RAM or HDD. The most we’ve ever simulated just to see if it’d work was close to 700k. And it was a lag fest, simply due to the fact that we were running out of RAM on a 32GB system. By the end of a 60 minute benchmark, the system had written over 80GB to the HDD. But the CPU was never being fully utilized. We still had tons of room left for other features of a game.

Gah, Fourier transforms work wonders

Edit: Although it did work quite well on a 256GB distributed rack server. I don’t imagine everyone has one of those in their garages haha (**** that’d be nice)
Second Edit: To my knowledge though, Everquest Next has a similar system. The only thing we do better (Once again to my knowledge) is our LOD system seeing how I can confidently say that they need to do all their AI server side. I’m not ranting against them, I love all those dudes over there. Some of the coolest guys ever!

Oh I should also add that we aren’t looking for programmers anymore. Our team is in a good state, and if we keep our numbers we should be able to finish the game in a practical time (If we do it too fast, computers won’t be beefy enough for the minimum requirements)


Note to self, most applications can’t handle 2GB of scatterplot data… stares at excel

I was hopping that I could use excel to look at my graphs before the editor was finished. That didn’t work as well as I hoped.

The tutorial is being postponed until we finish up on our own systems within the next month. Sorry guys! I can still answer questions on it though if anyone has any