Announcement

Collapse
No announcement yet.

TensorFlow

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    #31
    Originally posted by envenger View Post
    I am getting the following error when opening a component after installing tensorflow 2.0
    Click image for larger version

Name:	Capture.PNG
Views:	131
Size:	167.2 KB
ID:	1605275
    open an issue at https://github.com/getnamo/tensorflow-ue4/issues and give detailed steps to reproduce behavior.
    Plugins: GES - Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo

    Comment


      #32
      I have an issue, no matter what i return, the answer is always None.
      Click image for larger version  Name:	Capture.PNG Views:	1 Size:	87.5 KB ID:	1610884

      How is https://github.com/getnamo/tensorflow-native-ue4 version of your plugin currently?

      I have a trained model but using python plugin is a huge pain to test.
      Each time i compile without closing the project, the socketIO plugin crashes the project.
      Last edited by envenger; 04-22-2019, 06:33 PM.

      Comment


        #33
        Originally posted by envenger View Post
        I have an issue, no matter what i return, the answer is always None.
        Click image for larger version Name:	Capture.PNG Views:	1 Size:	87.5 KB ID:	1610884

        How is https://github.com/getnamo/tensorflow-native-ue4 version of your plugin currently?

        I have a trained model but using python plugin is a huge pain to test.
        That function doesn't return as you think. The python TensorflowComponent wraps callbacks such that they will automatically call back on json_input_gt_callback whether you use multi-threading or not. If you do have multi-threading on you wouldn't be able to receive the answer within a function callback anyway. You need to listen to json_input_gt_callback function which has the json results you're looking for. See https://github.com/getnamo/tensorflo...ponent.py#L101 for the python logic handling this. You can modify that section to return the results directly if you don't use multi-threading.

        I generally haven't used this plugin with c++ inference, typically developing and calling json input from BP is more amenable to ML prototyping. That said I think a refactor is in order which will allow the tensorflow component to be called natively which would simplify cases like these (and using the same api to call remote python servers), this refactor may be a while though as I don't have free opensource time in the near term.

        Originally posted by envenger View Post
        Each time i compile without closing the project, the socketIO plugin crashes the project.
        Any logs from such crashes would be welcome as a new issue under https://github.com/getnamo/socketio-client-ue4/issues, please give a detailed repro if possible.

        Originally posted by envenger View Post
        How is https://github.com/getnamo/tensorflow-native-ue4 version of your plugin currently?
        ​​​​​​​
        Same issue with time currently, but you can try using the c_api natively right now (try any https://github.com/Neargye/hello_tf_c_api, e.g. https://github.com/Neargye/hello_tf_...ession_run.cpp), it just doesn't have any ue4 specific data formats or conveniences implemented. If you get some good workflow going consider contributing.
        Plugins: GES - Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo

        Comment


          #34
          I'm getting an issue of ImportError: DLL load failed: The specified module could not be found. similar to getnamo above, however I have tensorflow gpu successfully installed on my computer. I installed tensorflow with Anaconda. Could that have something to do with the plugin not finding it?

          Comment


            #35
            Originally posted by StevePeters View Post
            I'm getting an issue of ImportError: DLL load failed: The specified module could not be found. similar to getnamo above, however I have tensorflow gpu successfully installed on my computer. I installed tensorflow with Anaconda. Could that have something to do with the plugin not finding it?
            The plugin is using an embedded python setup, so a separate tensorflow installation is managed by the plugin. What you do need to do is ensure you've installed the cuda and cudNN pre-requisites for gpu https://www.tensorflow.org/install/gpu#windows_setup. Please check https://github.com/getnamo/tensorflo...llation--setup
            Plugins: GES - Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo

            Comment


              #36
              I managed to get a TF2 version of addExample.py working, but I'm a bit confused on how to correctly implement
              the operation self.c = self.a + self.b

              Code:
              import tensorflow as tf
                import unreal_engine as ue
                from TFPluginAPI import TFPluginAPI
                  classExampleAPI(TFPluginAPI):
                  #expected optional api: setup your model for training
                defonSetup(self):
                  self.a = tf.Variable([0.0], tf.float32)
                self.b = tf.Variable([0.0], tf.float32)
                  self.op = tf.Variable(True, tf.bool)
                pass
                
                #expected optional api: parse input object and return a result object, which will be converted to json for UE4
                defonJsonInput(self, jsonInput):
                
                print(jsonInput)
                  self.a = tf.dtypes.cast(jsonInput['a'], tf.float32)
                self.b = tf.dtypes.cast(jsonInput['b'], tf.float32)
                  ifself.op:
                return tf.add(self.a, self.b).numpy().tolist()
                
                else:
                return tf.subtract(self.a, self.b).numpy().tolist()
                  
                
                  #custom function to change the op
                defchangeOperation(self, type):
                if(type=='+'):
                self.op =True
                  elif(type=='-'):
                self.op =False
                  defgetVersion(self, jsonInput):
                          ver = tf.__version__
                print(ver)
                return("GPU Available: ", tf.test.is_gpu_available())
               
              #expected optional api: start training your network
                defonBeginTraining(self):
                pass
                 
                #NOTE: this is a module function, not a class function. Change your CLASSNAME to reflect your class
                #required function to get our api
                defgetApi():
                #return CLASSNAME.getInstance()
                return ExampleAPI.getInstance()

              Comment


                #37
                Originally posted by getnamo View Post

                The plugin is using an embedded python setup, so a separate tensorflow installation is managed by the plugin. What you do need to do is ensure you've installed the cuda and cudNN pre-requisites for gpu https://www.tensorflow.org/install/gpu#windows_setup. Please check https://github.com/getnamo/tensorflo...llation--setup
                Love your plugin. I'm thinking about making something quite similar that merely visualizes different information from TensorFlow in an Unreal Engine 3D world.

                I don't know exactly how it'll look yet, but some combination of these two videos:

                https://www.youtube.com/watch?v=3JQ3hYko51Y

                https://www.youtube.com/watch?v=AgkfIQ4IGaM

                I've looked pretty hard and haven't seen anything close to what I'm thinking of building. If I move forward with it I think your plugin will help jumpstart my progress, thanks!

                Comment


                  #38
                  Originally posted by jimmyw_404 View Post
                  Love your plugin. I'm thinking about making something quite similar that merely visualizes different information from TensorFlow in an Unreal Engine 3D world.

                  I don't know exactly how it'll look yet, but some combination of these two videos:

                  https://www.youtube.com/watch?v=3JQ3hYko51Y

                  https://www.youtube.com/watch?v=AgkfIQ4IGaM

                  I've looked pretty hard and haven't seen anything close to what I'm thinking of building. If I move forward with it I think your plugin will help jumpstart my progress, thanks!
                  Sounds like a cool project! The only thing I'm aware of that is similar using this plugin is this german thesis work by Marcel Bock https://www.semanticscholar.org/pape...eefdf65d362b10, article: https://www.dlr.de/sc/en/desktopdefa...74_read-53313/

                  Might want to take a look for inspiration.
                  Plugins: GES - Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo

                  Comment


                    #39
                    Plugin Architecture Update
                    Working on an API refactor for this plugin to open up more backends and development environments, see https://github.com/getnamo/tensorflow-ue4/issues/53 for the parent issue tracking various parts of this effort.

                    The general idea is that you should be able to easily boot up a server (local or truly remote) and do remote dev work as if it was running UnrealEnginePython allowing you to train in more typical linux environments with no restrictions on either machine learning library or versions. The matching unreal frontend would be very lightweight and should be available on most os platforms. This should reduce setup, mismatch, and version headaches. The tensorflow-ue4 (unrealenginepython environment) plugin would still be one of the possible backends and the api will be very similar to the old one, just with the option of swapping it for a remote or a native variant without other code change.

                    In addition there is planned work on the tensorflow-native-ue4 plugin to use similar base api as the remote/python one, but with an inference focus. This would enable you to package e.g. .pb file and run inference on your trained model at native speeds. There is a possibility to expand this API to support more than inference directly from BP, but it wouldn't be the focus at this time.

                    If you're using the library, feedback on this new architecture work is welcome as I want to make sure it covers use cases you'd be interested in.
                    Last edited by getnamo; 12-02-2019, 03:14 PM.
                    Plugins: GES - Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo

                    Comment


                      #40
                      This is awesome. I want to train different with AIs with different Game Lore/World Datasets to drive NPC Chatbots. I'm open to any conversation on this topic.
                      GOD of DREAMs: A Unique FTPS/RPG Sandbox MMO

                      Comment


                        #41
                        Machine Learning Remote Plugin (remote variant of tensorflow-ue4) Released.

                        The remote variant of the plugin is now functional, check out: https://github.com/getnamo/machine-learning-remote-ue4 for the plugin and https://github.com/getnamo/ml-remote-server for the complement server. The plugin should be usable in a much wider variety of platforms (anywhere the socket.io plugin can be used so: win, linux, mac, android, ios) and you can host your server part anywhere that supports python 3 (typically linux, win, mac).

                        If you wanted a lighter client or a more flexible backend this should be the one to check out. Let me know if you use it and if you're missing some API.
                        Plugins: GES - Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo

                        Comment


                          #42
                          Originally posted by TechLord View Post
                          This is awesome. I want to train different with AIs with different Game Lore/World Datasets to drive NPC Chatbots. I'm open to any conversation on this topic.
                          You would probably be interested in models like gpt-2 https://openai.com/blog/better-language-models/ which now has the full 1.5B parameter model released: https://openai.com/blog/gpt-2-1-5b-release/ (it's super heavy though). This can predict fairly decently additional text based on a snippet of input text. It can even answer some questions about a text (reading comprehension).

                          Below was an example I used for testing reading comprehension. Everything below SAMPLE 1 is generated by gpt-2 in response to the text above it.
                          Plugins: GES - Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo

                          Comment


                            #43
                            getnamo Thanks for your insight.

                            Originally posted by getnamo View Post

                            You would probably be interested in models like gpt-2 https://openai.com/blog/better-language-models/ which now has the full 1.5B parameter model released: https://openai.com/blog/gpt-2-1-5b-release/ (it's super heavy though). This can predict fairly decently additional text based on a snippet of input text. It can even answer some questions about a text (reading comprehension).

                            Below was an example I used for testing reading comprehension. Everything below SAMPLE 1 is generated by gpt-2 in response to the text above it.
                            GOD of DREAMs: A Unique FTPS/RPG Sandbox MMO

                            Comment

                            Working...
                            X