Neural Network - A journey into the mind

Community Showcase Game Development
1

Hi everyone, I have been hard at work for the past few days working on something i am really passionate about. Neural networks. I made my own reinforcement learning algorithm a while back to play tappy chicken and it inspired me to make more things like this. So I purchased the basic neural network from the marketplace and i started messing with it to see how it performed. I immediately noticed if you throw a a couple hundred neural network blueprints in the world, it will freeze or crash. Sad day, so i decided to look at the project and tried to develop a c++ implementation, i tried and failed, i got very close to implementing it, one to one. but i just couldn’t figure his inheritance NN implementation out to correctly give an output. i even sent the original marketplace projects maker the c++ code, to get information on why but its pretty big and might take some time. but i got impatient just created my own non inheritance based C++ neural network from scratch.

this thread will serve as some place to throw all of my work on the plugin at.
I want to implement some cool things i think many people might find really interesting.

----Information about the plugin----
It has a Feed Forward and Back propagation Neural Network
It has things like various learning types, Supervised, Unsupervised, Reinforcement(negative and positive), as well as Evolved Genetic Neural Networks.

Its a only a C++ plugin, written by me with all 100% UE4 code. No external libraries used. I even wrote my own hyperbolic functions!

so for my first example i am showcasing race results of two types of neural networks VS each other and VS a generic racing checkpoint following AI.
I found a 2D racing template and i implemented two Racing AI’s to showcase the networks.
you can find the 2d racing template here

https://youtube.com/watch?v=NtsJf0ottIw
Video information
Blue cars- Simple Checkpoint following AI which came with the 2D Racing template.
Yellow cars- Simple Neural Network AI, Racer Avoidance. Checkpoint following AI(Constant racer neural parameters, like, sight distance, sight fov, breaking and accelerating aggressiveness, non-variable neural network hidden layer counts and number of hidden layers 3 inputs, 5 hidden, 10 hidden, 5 hidden 3 hidden, 2 output)
Red cars- Advanced Genetic Neural Network AI, Individual Racer Avoidance. Checkpoint following AI.(Genetic racer neural parameters, like, sight distance, sight fov, breaking and accelerating aggressiveness, variable neural network hidden layer counts and number of hidden layers 3 inputs, rand(x) hidden, for y, 2 output)
The drivers were all either trained or evolved and trained before the video took place.
The cars all have the same attributes, only the driver’s abilities change.

----What’s to come?----
I’m thinking about a maybe a neural network to operate as a post process as an end goal. but i don’t know how to even start that. im looking in to it though as there are really cool ideas you can use.
So im going to look in to other gameplay ai’s like maybe the classification of a players state, like if they are healthy or not. that sounds pretty cool and an easy task to teach how to use the plugin.

2

I made the neural networks be able to forward-propagate and backward-propagate in a separate thread. Right now this creates a thread for each individual NN, so you can split supervised training data into separate threads. Im also working on Batch NN threading (n NN’s per thread) as well as All NN in a separate single thread.

1 Threaded NN with supervised learning VS Non-Threaded NN with supervised learning split across 4 threads are about 3 to 4x faster, its about 3.5 to 1, speedup increase. as expected since i have a quad core.
Threaded NN’s with unsupervised learning VS Non-Threaded NN’s with unsupervised learning are not any faster at all, its about 1 to 1, as expected.

Sometimes Unsupervised Threaded NN are slower when you have say 300 NN’s because the sheer amount, since threaded are doing more work because the creation of threads /copying data ect. and the kind of unsupervised learning too takes a toll as it has to generate it’s own optimal outputs as well as compute the final outputs vs only having to compute the final outputs in supervised…

Edit: I have a feeling im going to remove threading altogether as there is no real reason for it to be there other than to speed up supervised learning, which to me is a very specific optimization, for a very general ai. ill come back to the threaded code at a later date, when i can find a use for the speed.

3

I have been waiting for this for so long! Looks great! Can I see your tappy bird project?

4

I’m going to be releasing the my neural networked tappy bird project and the neural network plugin in conjunction at a future undetermined date. It will be relatively soon. I have to do some spring cleaning on both the example project and the network but it will get there and it will be glorious. And free.

Edit: Holy nice! I was Spotlit! Thank you Alexander and Epic!!!

Edit2: I might just submit the base nn plugin to epic as a pr for the release branch, idk if they accept plugin pr’s but i think it would be better suited in the engine, we could use it for neural net anticheat and such, endless possibilities actually.

5

Awesome, cant wait to give this a whirl

6

An anticheat would actually be really cool. But to be honest, what other applications do machine learning techniques have in the gaming industry? Using them for ai tends not to make much sense. I’d love to work on an anticheat though.

7

Honestly there are so many applications it’s kinda like asking what is programming good for. I showed for example, the ai - path following and it will optimize itself, speed adjustment and it will optimize itself, you could just as easily adapt this to anything you’re mind desires. for example, checkers, blackjack, player spoken actions(speech to commands or speech to text). the ideas are only limited by you. NN’s are essentially black boxes that take input and give an output based on some neuron layer setup which is the black box.

You can just make a general ai that can just straight up try and play any game as a human would based trial and error, trained only on a small player input dataset. This is the end goal of this tool. of course simple games are better suited for a nn like this but nn’s can do very well in many different spaces. Of course this is yet to be supported yet as i haven’t had the time to expand on my current work of convolutions. convolutions are a fancy way of saying ‘getting a nn to learn to see pictures’. but not really. see http://i.stack.imgur.com/I7DBr.gif its much more complicated than that. but for example You could have an ai that looked out in to the world and tried to “facial recognize” player characters. This is computationally expensive if were done real time, but solutions can be baked before hand. So while the developer has to bake some information, it only has to be done once ever per training set and can be distributed to clients and colleagues. What this kind of ai could be useful for is anything you could think of using an ai that can learn what a player face is, but for example, in an fps game, you could train it before hand on ally faces and enemy faces and have the ai neural net learn what is friendly and not friendly and have the convolution NN scan a portion of their view pasted onto a rendertarget for enemy faces, and guess if its enemy or friendly, if it thinks its an enemy, shoot them dead. this will inevitably have some quirks though like the NN shooting and things that are not characters at all, NN shooting at allies, those two are okay as we are trying to replicate human brains. but things like down right owning humans completely 100% can happen in a well trained net.

this may have just *convoluted *things idk, i tried my best XD

as for the anti cheat i have two ideas.
i am thinking about using a neural network to learn from players mouse’s heat-map to learn what could be inhuman movement. (like perfectly horizontal, diagonal, vertical movements). this could be potentially ground breaking for aimbot like cheats.

the other idea i had was training a nn on watching confirmed cheat game demos and have it try and tell if a player was cheating or not. for things like walls. idk if this will work though. but i want to try.

8

@SaxonRah Although all of those ideas are really cool, it does not make sense to use neural networks for most of those systems. Neural networks would work, but there are much easier ways to get very similar results.

I think adding bayesian networks would be really useful. Bayesian networks are excellent when it comes to procedural generation.

I think anticheat is the best application for games. ANN are perfect.

9

Well its not really the point that they make sense, its more the point of creating semi human like ai that is more believable, than a straight statistical behavioral tree or what have you. most ai systems are purely statistical mathematical models that are very robotic.

I’ve added Bayesian networks and hopfield networks are on the to do list!

10

This is cool, I wrote a paper based on NNs awhile back and ever since I wanted to try my hand at creating one. Making one in unreal is awesome for anything thinkable! :slight_smile:

11

Any updates?

12

I’m working on it as fast as i can. I had to talk to a few people about licences of code before i released it, because while i did program everything from scratch using ue4 by myself, i did use public information and wanted to make sure i wasn’t doing anything illegal so i stopped working on it for a month while i hashed out the legality. So far so good and im back working on it. the MachineLearning Plugin and example project should be on github soon, like maybe a week or two?

13

A little bit of an update. Release ! Yay

A little bit of an update. I decided to change the goal and scope of this plugin/journey/whatever the heck this is.

What im going to be doing is using the NN to predict poker hands, with generative data as the training data. This means we will be simulating poker games. getting hands, using an analyzer to figure out poker hand rank by bucket counting. then feeding this as training data.
The learning is supervised type, because we are using labeled training data. Although the data is generative, it doesn’t matter in the end. We could have used poker hand data but then you have to transform the data into useable data, i opted not to use predefined data.

I have the neural network plugin done. I have a deck of cards plugin done, a few simple pokergame functions to test and analyze the cards. I have everything needed to get the ANN to classify a hand of cards. Except the implementation of the interconnectivity between the cards and the net. you know, the fun fiddly bits.
Here is the link for the github

For the base project, which is the BP Blackjack project for assets. Although in the end we will cut ties with all assets in the future. As a result, there is a new card mesh with new textures and mats. Backs and faces included.
The deck of cards and test poker game plugin.
the Artificial Neural Network plugin.

Please enjoy and, please notify me of anything wrong that you might see.

MachinaPlays_SS3.jpg
MachinaPlays_SS3.jpg2821×1547 524 KB
&stc=1&d=1471371904
MachinaPlays_SS2.png
MachinaPlays_SS2.png1287×716 93.9 KB
&stc=1&d=1471371947
C9NAat.jpg
C9NAat.jpg1611×942 345 KB
&stc=1&d=1471372030

14

I’m a little confused what the goal is.

15

To classify poker hands. it’s a simple but easy to understand example of how to use a NN to someone who doesn’t know anything

16

Alright sweet. I’ll definitely download it once I return from vacation in a couple weeks. Keep it up!!!

17

I just committed a basic test of the plugins working together. :cool:

18

I released a new update, this one has generative training data test, + helped generative training data

19

I just updated the github with a major bug fix that made one of the neural networks result in more correct than wrong after some training More work to be done on the other though, think i have a bug somewhere.

20

Hey,

Seems a great plugin, but im having trouble getting it working with 4.13.1.

Do you know if it works ok with that version?

Cheers.