Is it possible to make a game ai that learns and adapts?

I was just thinking about all the advancements in robotics and artificial intelligence recently, and was wondering if it could be applied to the gaming industry.

Absolutely. We are working on something for our fighting game that does just that. Though probably not at an advanced of a level you are referring to. This a quick free abstract, but there are tons of books on the subject, every one of them awesome.

I remember back in 1999 there was a Quake 2 bot that used neural networks to learn the game. It was never good enough to actually provide any kind of challenge for players, but it shows how long this stuff has been around. It’s just not used much because it’s quite a lot of effort and, in most shooters, the player won’t notice the difference. It also gives you less control over the difficulty curve of your game if the AI is changing as it goes along.

The best Quake bot was Eraser Bot … it learnt where the best weapons were from the players and kept it in its route. You could also teach the bot by running a console command and then running through the map … this created the route for the bot.

The more you played the map with the bots the more sneaky they got and could actually pose a challenge … if you had a bigger weapon and shot them, they would run away to look for a bigger weapon … it was pretty cool.

I guess the trick here is going to be how do you store this learning information for the bots over various maps/levels … I think once you nail that, you could make the bot learn anything and become sneaky.

ANN’s are the way to go with this I recon…

Why doesn’t UE4 have a UTBot like in UDK.

I would love to go against that ******* again. LOL. :smiley:

Epic, please add to next update.

This has already been done since 1996. The reason why it’s not more prevalent is because many gamers can’t handle such AI. AI in games are purposely dumbed down to increase end-user enjoyment and profit.

Here’s a link.

“Most players still tunnel visioned in on the bait like tween girls on Edward Cullen.”


That article is really interesting. It brings up a good point that players don’t want smarter ai, but ai with more variety. I really think a combination of the two would be the best.

Hmmm … that is food for thought … I better send this link to my one dev who is doing the AI for our game. 8-}

I think the UTBot code was lifted from the source to UT3 (along with all the other UT* classes). Since UE4 is still in development, they probably don’t have any source code ready yet. I’m sure someone will throw together an AI template for others to use.

I’m a bit of an AI fan myself, maybe I’ll give it a try when I have time :slight_smile:

#1. UE4 has no built in Bots yet??? Say it ain’t so…

#2. Thanks for the article! It had a boatload of common sense and little jargon, and shows how easy it is to forget about the obvious and lose track of what’s important :-

    • “Ask a Game Dev — Game Development Myths: Players Want Smart Artificial Intelligence - Because this is the only tactic that worked well consistently, the players complained that the AI was dumb for just running in and getting shot while they stayed in relative safety. - In essence, we had built an AI that really did outsmart most players and players really don’t respond well to being outsmarted, so they had turned the tables by repeatedly dragging the AI into situations where it was dumbest, and then winning through attrition. And that was a problem, because it wasn’t fun. But it sure was effective.”
  • “That is not what the goal should have been. The goal from the start should have been to design an AI that was fun to beat. This is a rather crucial difference that I think we glossed over at the beginning, and it ended up hurting us in the long run. In retrospect, had we been able to do it over again, I would have erred on the side of more variety and less “intelligence”. If we had enemies who actually behaved differently it would then be up to the player to mentally come up with a strategy to deal with each (recognizable) type and thus improve and exhibit skill with the game. As long as we provided a steady stream of reasonably interesting and varied decisions and choices to make (a mix of snipers, assault types, pistol types, breachers, mid-long range rifles, etc.) then the gun combat would be more interesting and less frustrating. It may not have been the “smart” AI that some players clamor for, but it would have been fun to play against and beat, and that’s what really matters in the end.”*

This article talks about single player AI, where the player is usually outnumbered by the NPCs. In this scenario I agree that the AI cannot be too smart and effective, otherwise the game is too difficult. The AI is supposed to simulate goons that the player can take on 3 at a time, not smart humans for 1 v 1 combat.

With a game like UT however, the AI is supposed to replace the human players you’d normally fight online, so it is in your interest to make them as smart as possible so that they can effectively fill in for a human.

Interesting read if you haven’t seen it before :-

  • In UDK we had Bots that could exhibit behaviours, but they never really lived up to potential and were often buggy. I remember thinking it would be interesting to design a Bot that could exhibit human emotions, aka Emotioneering in Video Games. That was a big deal a decade ago post GTA Vice City, when notable screenwriters were sucked into video games companies. But did it ever go anywhere?

  • For example, what if a couple of bots pin you down i.e. Bots exhibiting confidence (aggressive pursuit / accuracy of fire). But then you get lucky and manage to kill one and wound the other. The remaining Bot takes some more pot-shots but misses. What then? Well, what if the Bot became scared (survival fear), and chose to run off or even surrender sometimes…

  • Stuff like this seems obvious, but you don’t see it much in well known games, unless I’m mistaken? There’s always lots of talk about adding real-life human emotions to Bot AI at game conferences. But after you play the game, it often feels more like just another type of cover system.

  • Overall, I think one of the core problems with video-game goals sometimes, is that it just takes so much work and time to put the core level(s) together, get the weapons, vehicles, characters working, and polish the materials / textures, GUI, AI etc etc, i.e. the basic mechanics, that its easy to lose perspective on the distinctive things that were planned for the game from the start…

  • What do others think about this?

I’m no AI expert (Just a designer) but I’d basically mimic Cleverbot. Create a matrix of real person responses to various situations and base my move off that (or similar situations). Every game adds to the knowledge matrix, and when you go “off book” (a chess term) you’d revert back to that baseline AI. I’d also expose what is going on behind the scenes to players, so they can see how the game is “thinking” - Could be a fun gimmick. It isn’t “real” learning in the AI sense, but it could emulate that pretty well I think, including making mistakes! Would work best in a turn based game obviously.


I think one day soon someone (or some team) will program an ai system that blows everything else out of the water. For some reason I feel many game devs look at ai the wrong way. Maybe it is because fps ai hasn’t really changed as much over the last 10 years as I would have expected it to. Just some food for thought.

I think we’re coming back around to the practicality vs. complexity argument again. Having bots that simulate human emotion and feed that back into their playing is great if you’re doing a research project, or you want some fancy marketing to put on the back of the box, but it’s not very practical from a gameplay perspective. When playing online you never consider the other person’s mood and emotions because usually you can’t tell as you’re pumping them full of lead unless they’re crying about their recent break-up on teamspeak. Real humans base their decision making on thousands upon thousands of tiny inputs, too many to attempt simulating in AI. Perhaps you missed the bus and stepped in a puddle on your way home, so you’re in a bad mood, which affects your decision-making in some subtle way. Ultimately though, it boils down to:

  • What is my opponent’s current tactic when fighting?
  • How do I think they’ll respond if I take a certain action?
  • How accurate are they?

Of course, team games like CTF, Onslaught, Assault etc. offer up a wealth of tactical opportunities which can be exploited (for example, waiting in ambush for the flag carrier rather than charging in, use of specific vehicles etc). However, this is less about emotion and more about your understanding of your opponent and how you think they’re likely to react to certain actions, planning ahead and deriving a prediction based on past experience. It’s nice to have bots that learn, but if you can achieve the same (or almost identical) effect with a fraction of the effort by pre-programming certain tactics instead, then that is clearly a better option when you’re developing a game.

This, I think, is why developers sometimes over-think things. The AI is part of the gameplay, and it should receive the same treatment as any other gameplay mechanic. It must fit with the rest of the design, should be lean (no superfluous features that have no impact on the player’s experience) and most of all, should be fun! has all the skinny on the current work happening on videogame AI.

Long story short: There’s a ton of techniques, some of which work and some of which require more thinking. Machine learning was a big thing a few years ago and there was a lot of discussion about its applicability, but for the most part AI for shooter type games still uses what we call reactive planners. I.E behavior tree’s and their ilk.

You can do simple “learning” fairly easily, but its an ongoing area of AI research so you can go as far as you like really (plenty of PhD level work to do still in that area).

If you’re interested, there are a loads of competitions for game AI being run. One of them was an Unreal Bot competition. So maybe google the results of that.

A part of me feels like smart AI has taken a back-seat to online gaming. It’s like, why create smart AI that simulates human thinking when we can just get other humans. The only downside is that I like to role play in a game, and when the other guy is jump-shotting me all over the place, it sucks me out of the game.
I say to myself, “Why don’t you just play Quake?”

  • Dead on! Online gaming has become pretty stale and formulaic even with human players. Some examples from the Call-Of-Duty / Far Cry series which turned me off. First there’s the obvious problem of online cheats whose binary edits can get around punkbuster type cheater defences i.e. shooting through walls, walking through walls, moving at super-human speeds etc…

  • Then there’s the worst aspect: unrealistic gameplay abilities. Players with spacebar techniques that enable them to jump into the air three times in succession to avoid being shot at, while simultaneously killing everyone else. Yeah, that’s realistic! Its the same with deadly Terminator like aiming with impossibly accurate shots across entire maps while simultaneously running & jumping… Here, momentum-variables and virtual-wind should play a part, with the player failing to hit anything most of the time. But overall, these player traits should just be outlawed. Offer custom patches and more Server settings to ban this type of abuse…

  • Instead, introduce new tactics. What if driverless vehicles could be used as decoys (brick on accelerator etc). Has this been done?.. To rob a joke from Chris Rock bullets should cost thousands of dollars in-game too! That way they’d be more precious, and perhaps there’d be more tension instead of every player knowing they’ll just get respawn with a full clip in a moment anyway. In that sense, Search and Destroy is the more ‘honest’ gametype! Last but not least: Lag! While I’m no fan of golf it has a handicap system that works well to match players of unequal abilities and strengths. I’d like to see that implemented for lots of things including frag counts, but specifically it’d help compensate for millisecond lag, so that the fastest guy doesn’t always win because he’s the one with the fiber connection!