Announcement

Collapse
No announcement yet.

Neural Network - A journey into the mind.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    [PLUGIN] Neural Network - A journey into the mind.

    Hi everyone, I have been hard at work for the past few days working on something i am really passionate about. Neural networks. I made my own reinforcement learning algorithm a while back to play tappy chicken and it inspired me to make more things like this. So I purchased the basic neural network from the marketplace and i started messing with it to see how it performed. I immediately noticed if you throw a a couple hundred neural network blueprints in the world, it will freeze or crash. Sad day, so i decided to look at the project and tried to develop a c++ implementation, i tried and failed, i got very close to implementing it, one to one. but i just couldn't figure his inheritance NN implementation out to correctly give an output. i even sent the original marketplace projects maker the c++ code, to get information on why but its pretty big and might take some time. but i got impatient just created my own non inheritance based C++ neural network from scratch.

    this thread will serve as some place to throw all of my work on the plugin at.
    I want to implement some cool things i think many people might find really interesting.

    ----Information about the plugin----
    It has a Feed Forward and Back propagation Neural Network
    It has things like various learning types, Supervised, Unsupervised, Reinforcement(negative and positive), as well as Evolved Genetic Neural Networks.


    Its a only a C++ plugin, written by me with all 100% UE4 code. No external libraries used. I even wrote my own hyperbolic functions!

    so for my first example i am showcasing race results of two types of neural networks VS each other and VS a generic racing checkpoint following AI.
    I found a 2D racing template and i implemented two Racing AI's to showcase the networks.
    you can find the 2d racing template here
    https://forums.unrealengine.com/show...lueprint-Power


    Video information
    Blue cars- Simple Checkpoint following AI which came with the 2D Racing template.
    Yellow cars- Simple Neural Network AI, Racer Avoidance. Checkpoint following AI(Constant racer neural parameters, like, sight distance, sight fov, breaking and accelerating aggressiveness, non-variable neural network hidden layer counts and number of hidden layers 3 inputs, 5 hidden, 10 hidden, 5 hidden 3 hidden, 2 output)
    Red cars- Advanced Genetic Neural Network AI, Individual Racer Avoidance. Checkpoint following AI.(Genetic racer neural parameters, like, sight distance, sight fov, breaking and accelerating aggressiveness, variable neural network hidden layer counts and number of hidden layers 3 inputs, rand(x) hidden, for y, 2 output)
    The drivers were all either trained or evolved and trained before the video took place.
    The cars all have the same attributes, only the driver's abilities change.

    ----What's to come?----
    I'm thinking about a maybe a neural network to operate as a post process as an end goal. but i don't know how to even start that. im looking in to it though as there are really cool ideas you can use.
    So im going to look in to other gameplay ai's like maybe the classification of a players state, like if they are healthy or not. that sounds pretty cool and an easy task to teach how to use the plugin.
    Last edited by SaxonRah; 06-16-2016, 04:58 AM.
    Youtube
    Machine Learning C++ Plugin
    Lindenmayer System C++ Plugin

    #2
    I made the neural networks be able to forward-propagate and backward-propagate in a separate thread. Right now this creates a thread for each individual NN, so you can split supervised training data into separate threads. Im also working on Batch NN threading (n NN's per thread) as well as All NN in a separate single thread.

    1 Threaded NN with supervised learning VS Non-Threaded NN with supervised learning split across 4 threads are about 3 to 4x faster, its about 3.5 to 1, speedup increase. as expected since i have a quad core.
    Threaded NN's with unsupervised learning VS Non-Threaded NN's with unsupervised learning are not any faster at all, its about 1 to 1, as expected.

    Sometimes Unsupervised Threaded NN are slower when you have say 300 NN's because the sheer amount, since threaded are doing more work because the creation of threads /copying data ect. and the kind of unsupervised learning too takes a toll as it has to generate it's own optimal outputs as well as compute the final outputs vs only having to compute the final outputs in supervised...



    Edit: I have a feeling im going to remove threading altogether as there is no real reason for it to be there other than to speed up supervised learning, which to me is a very specific optimization, for a very general ai. ill come back to the threaded code at a later date, when i can find a use for the speed.
    Last edited by SaxonRah; 06-16-2016, 07:12 AM.
    Youtube
    Machine Learning C++ Plugin
    Lindenmayer System C++ Plugin

    Comment


      #3
      I have been waiting for this for so long! Looks great! Can I see your tappy bird project?

      Comment


        #4
        Originally posted by phantom530 View Post
        I have been waiting for this for so long! Looks great! Can I see your tappy bird project?
        I'm going to be releasing the my neural networked tappy bird project and the neural network plugin in conjunction at a future undetermined date. It will be relatively soon. I have to do some spring cleaning on both the example project and the network but it will get there and it will be glorious. And free.


        Edit: Holy nice! I was Spotlit! Thank you Alexander and Epic!!!

        Edit2: I might just submit the base nn plugin to epic as a pr for the release branch, idk if they accept plugin pr's but i think it would be better suited in the engine, we could use it for neural net anticheat and such, endless possibilities actually.
        Last edited by SaxonRah; 07-05-2016, 06:43 PM.
        Youtube
        Machine Learning C++ Plugin
        Lindenmayer System C++ Plugin

        Comment


          #5
          Awesome, cant wait to give this a whirl

          Comment


            #6
            Originally posted by SaxonRah View Post
            I'm going to be releasing the my neural networked tappy bird project and the neural network plugin in conjunction at a future undetermined date. It will be relatively soon. I have to do some spring cleaning on both the example project and the network but it will get there and it will be glorious. And free.


            Edit: Holy nice! I was Spotlit! Thank you Alexander and Epic!!!

            Edit2: I might just submit the base nn plugin to epic as a pr for the release branch, idk if they accept plugin pr's but i think it would be better suited in the engine, we could use it for neural net anticheat and such, endless possibilities actually.

            An anticheat would actually be really cool. But to be honest, what other applications do machine learning techniques have in the gaming industry? Using them for ai tends not to make much sense. I'd love to work on an anticheat though.

            Comment


              #7
              Originally posted by phantom530 View Post
              An anticheat would actually be really cool. But to be honest, what other applications do machine learning techniques have in the gaming industry? Using them for ai tends not to make much sense. I'd love to work on an anticheat though.
              Honestly there are so many applications it's kinda like asking what is programming good for. I showed for example, the ai - path following and it will optimize itself, speed adjustment and it will optimize itself, you could just as easily adapt this to anything you're mind desires. for example, checkers, blackjack, player spoken actions(speech to commands or speech to text). the ideas are only limited by you. NN's are essentially black boxes that take input and give an output based on some neuron layer setup which is the black box.

              You can just make a general ai that can just straight up try and play any game as a human would based trial and error, trained only on a small player input dataset. This is the end goal of this tool. of course simple games are better suited for a nn like this but nn's can do very well in many different spaces. Of course this is yet to be supported yet as i haven't had the time to expand on my current work of convolutions. convolutions are a fancy way of saying 'getting a nn to learn to see pictures'. but not really. see http://i.stack.imgur.com/I7DBr.gif its much more complicated than that. but for example You could have an ai that looked out in to the world and tried to "facial recognize" player characters. This is computationally expensive if were done real time, but solutions can be baked before hand. So while the developer has to bake some information, it only has to be done once ever per training set and can be distributed to clients and colleagues. What this kind of ai could be useful for is anything you could think of using an ai that can learn what a player face is, but for example, in an fps game, you could train it before hand on ally faces and enemy faces and have the ai neural net learn what is friendly and not friendly and have the convolution NN scan a portion of their view pasted onto a rendertarget for enemy faces, and guess if its enemy or friendly, if it thinks its an enemy, shoot them dead. this will inevitably have some quirks though like the NN shooting and things that are not characters at all, NN shooting at allies, those two are okay as we are trying to replicate human brains. but things like down right owning humans completely 100% can happen in a well trained net.

              this may have just convoluted things idk, i tried my best XD


              as for the anti cheat i have two ideas.
              i am thinking about using a neural network to learn from players mouse's heat-map to learn what could be inhuman movement. (like perfectly horizontal, diagonal, vertical movements). this could be potentially ground breaking for aimbot like cheats.

              the other idea i had was training a nn on watching confirmed cheat game demos and have it try and tell if a player was cheating or not. for things like walls. idk if this will work though. but i want to try.
              Last edited by SaxonRah; 07-06-2016, 01:20 AM.
              Youtube
              Machine Learning C++ Plugin
              Lindenmayer System C++ Plugin

              Comment


                #8
                [MENTION=94]SaxonRah[/MENTION] Although all of those ideas are really cool, it does not make sense to use neural networks for most of those systems. Neural networks would work, but there are much easier ways to get very similar results.

                I think adding bayesian networks would be really useful. Bayesian networks are excellent when it comes to procedural generation.

                I think anticheat is the best application for games. ANN are perfect.

                Comment


                  #9
                  Originally posted by phantom530 View Post
                  @SaxonRah Although all of those ideas are really cool, it does not make sense to use neural networks for most of those systems. Neural networks would work, but there are much easier ways to get very similar results.

                  I think adding bayesian networks would be really useful. Bayesian networks are excellent when it comes to procedural generation.

                  I think anticheat is the best application for games. ANN are perfect.
                  Well its not really the point that they make sense, its more the point of creating semi human like ai that is more believable, than a straight statistical behavioral tree or what have you. most ai systems are purely statistical mathematical models that are very robotic.

                  I've added Bayesian networks and hopfield networks are on the to do list!
                  Last edited by SaxonRah; 07-06-2016, 01:16 PM.
                  Youtube
                  Machine Learning C++ Plugin
                  Lindenmayer System C++ Plugin

                  Comment


                    #10
                    This is cool, I wrote a paper based on NNs awhile back and ever since I wanted to try my hand at creating one. Making one in unreal is awesome for anything thinkable!
                    Rule#117: "Be polite, be professional, but have a plan to kill everyone you meet."

                    Comment


                      #11
                      Originally posted by SaxonRah View Post
                      I'm going to be releasing the my neural networked tappy bird project and the neural network plugin in conjunction at a future undetermined date. It will be relatively soon. I have to do some spring cleaning on both the example project and the network but it will get there and it will be glorious. And free.


                      Edit: Holy nice! I was Spotlit! Thank you Alexander and Epic!!!

                      Edit2: I might just submit the base nn plugin to epic as a pr for the release branch, idk if they accept plugin pr's but i think it would be better suited in the engine, we could use it for neural net anticheat and such, endless possibilities actually.
                      Any updates?

                      Comment


                        #12
                        Originally posted by phantom530 View Post
                        Any updates?
                        I'm working on it as fast as i can. I had to talk to a few people about licences of code before i released it, because while i did program everything from scratch using ue4 by myself, i did use public information and wanted to make sure i wasn't doing anything illegal so i stopped working on it for a month while i hashed out the legality. So far so good and im back working on it. the MachineLearning Plugin and example project should be on github soon, like maybe a week or two?
                        Last edited by SaxonRah; 08-03-2016, 10:43 AM.
                        Youtube
                        Machine Learning C++ Plugin
                        Lindenmayer System C++ Plugin

                        Comment


                          #13
                          A little bit of an update. Release ! Yay

                          A little bit of an update. I decided to change the goal and scope of this plugin/journey/whatever the heck this is.


                          What im going to be doing is using the NN to predict poker hands, with generative data as the training data. This means we will be simulating poker games. getting hands, using an analyzer to figure out poker hand rank by bucket counting. then feeding this as training data.
                          The learning is supervised type, because we are using labeled training data. Although the data is generative, it doesn't matter in the end. We could have used poker hand data but then you have to transform the data into useable data, i opted not to use predefined data.

                          I have the neural network plugin done. I have a deck of cards plugin done, a few simple pokergame functions to test and analyze the cards. I have everything needed to get the ANN to classify a hand of cards. Except the implementation of the interconnectivity between the cards and the net. you know, the fun fiddly bits.

                          Here is the link for the github


                          For the base project, which is the BP Blackjack project for assets. Although in the end we will cut ties with all assets in the future. As a result, there is a new card mesh with new textures and mats. Backs and faces included.
                          The deck of cards and test poker game plugin.
                          the Artificial Neural Network plugin.

                          Please enjoy and, please notify me of anything wrong that you might see.


                          https://forums.unrealengine.com/atta...1&d=1471371904
                          https://forums.unrealengine.com/atta...1&d=1471371947
                          https://forums.unrealengine.com/atta...1&d=1471372030
                          Attached Files
                          Last edited by SaxonRah; 08-20-2016, 12:01 AM.
                          Youtube
                          Machine Learning C++ Plugin
                          Lindenmayer System C++ Plugin

                          Comment


                            #14
                            Originally posted by SaxonRah View Post
                            A little bit of an update. I decided to change the goal and scope of this plugin/journey/whatever the heck this is.


                            What im going to be doing is using the NN to predict poker hands, with generative data as the training data. This means we will be simulating poker games. getting hands, using an analyzer to figure out poker hand rank by bucket counting. then feeding this as training data.
                            The learning is supervised type, because we are using labeled training data. Although the data is generative, it doesn't matter in the end. We could have used poker hand data but then you have to transform the data into useable data, i opted not to use predefined data.

                            I have the neural network plugin done. I have a deck of cards plugin done, a few simple pokergame functions to test and analyze the cards. I have everything needed to get the ANN to classify a hand of cards. Except the implementation of the interconnectivity between the cards and the net. you know, the fun fiddly bits.

                            Here is the link for the github


                            For the base project, which is the BP Blackjack project for assets. Although in the end we will cut ties with all assets in the future. As a result, there is a new card mesh with new textures and mats. Backs and faces included.
                            The deck of cards and test poker game plugin.
                            the Artificial Neural Network plugin.

                            Please enjoy and, please notify me of anything wrong that you might see.
                            I'm a little confused what the goal is.

                            Comment


                              #15
                              To classify poker hands. it's a simple but easy to understand example of how to use a NN to someone who doesn't know anything
                              Youtube
                              Machine Learning C++ Plugin
                              Lindenmayer System C++ Plugin

                              Comment

                              Working...
                              X