Announcement

Collapse
No announcement yet.

TensorFlow

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    #16
    This is incredibly cool coding ! Can one use ue4 node editor as general c++ visual coding platform, by following this gourgeous implementation ? Is it basically about linking dll functions to the blueprint nodes ?

    Comment


      #17
      Originally posted by Metusalemski View Post
      This is incredibly cool coding ! Can one use ue4 node editor as general c++ visual coding platform, by following this gourgeous implementation ? Is it basically about linking dll functions to the blueprint nodes ?
      The unreal side for communicating data is typically done in blueprint (node editor) via a TensorflowComponent and the machine learning is done in python (which is what tensorflow uses so you can copy paste examples).


      Originally posted by AnaRhisT94 View Post

      Have you released the pong game yet? I'm currently developing a simple 2d game with a stick that tries to avoid enemies that are moving in the environment by pulling up/down the stick.

      Implemented it in a separate branch of the example project: https://github.com/getnamo/tensorflo...es/tree/qlearn. All the pieces are there, it's just missing a trained network to work in theory.
      Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

      Comment


        #18
        This is awesome!

        Comment


          #19
          Is tensorflow 2.0 supported and would I need to change something to get it download by pip?

          Comment


            #20
            Originally posted by envenger View Post
            Is tensorflow 2.0 supported and would I need to change something to get it download by pip?
            In theory just changing "tensorflow": "1.10.0", to "2.0.0" inside https://github.com/getnamo/tensorflo...upymodule.json should do what you're looking for, but there may be gremlins in the new tf API, kindly open a github issue with details (https://github.com/getnamo/tensorflow-ue4/issues) if you run into problems.
            Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

            Comment


              #21
              Hi there!
              I have a few newb questions about this (apologies):

              * Is there C/C++ API support? As in, do I have to use blueprints to communicate with the plugin, or can I just write code?
              * What format does a trained network take in this scenario- is it a .uasset? I'm more interested in using this library as a way to run already-trained tensor flow networks in engine rather than as a way to train, so packaging is an important question.
              * You mention only supporting Windows platform at this point. Is that for training only, or also for running a network? Theoretically it shouldn't be that big of a deal to get that aspect running on consoles? Or is this a dependency on the Python plugin?

              Thanks for your help understanding.
              -David

              Comment


                #22
                Originally posted by David.Bromberg View Post
                * Is there C/C++ API support? As in, do I have to use blueprints to communicate with the plugin, or can I just write code?
                The plugin is structured around a blueprint actor component called Tensorflow Component which wraps threading and communication to an embedded python layer. This way all of your machine learning can use largely unmodified tensorflow python files and on the unreal side you just have to worry about how to structure your data for your model. Basic usage instructions can be found here: https://github.com/getnamo/tensorflo...rflowcomponent

                Because the component is put together in blueprint it would be troublesome to use it in C++ directly, you may need write a wrapper that has a C++ base and is overwritten in blueprint. It would probably make sense to refactor the Tensorflow component into a C++ base so that it can be called from both ends (a good enhancement issue).

                Originally posted by David.Bromberg View Post
                * What format does a trained network take in this scenario- is it a .uasset? I'm more interested in using this library as a way to run already-trained tensor flow networks in engine rather than as a way to train, so packaging is an important question.
                The trained network will be your usual .pb or checkpoints as all your machine learning should be vanilla tensorflow. The plugin already does package correctly.

                Originally posted by David.Bromberg View Post
                * You mention only supporting Windows platform at this point. Is that for training only, or also for running a network? Theoretically it shouldn't be that big of a deal to get that aspect running on consoles? Or is this a dependency on the Python plugin?
                It is limited to windows atm, because it uses a cmd subprocess to handle pip dependencies without blocking anything, this can probably be expanded to multiple platforms (Source can be found https://github.com/getnamo/UnrealEng...ipts/upypip.py). The pip and python dependency would make console support hard atm.

                That said wider support has been on my mind, specifically inference.

                Originally I thought of including a tensorflow dll directly in this plugin, but the api was moving too rapidly back then and there were no dlls to download, so the python approach was chosen to allow for easy updates to latest builds. The scene has changed since then and while the repository does include a tensorflow dll, I believe it will probably be best to split native tensorflow into a fresh plugin that will be inference focused, something like tensorflow-native-ue4? It would probably use the c api with some c++ wrapper and simplify loading your model, running inference and getting data in ue4 format. I made a blank plugin just now for that: https://github.com/getnamo/tensorflow-native-ue4, but it will take some time to bring it to a functional build. Contributions welcome if you want to help, there would need to be native builds of the library for whatever target hardware you're looking for.

                Edit: https://github.com/getnamo/tensorflow-native-ue4 is now functional if you want to use the tensorflow c_api directly, but there are no convenience unreal type bindings yet. Will come back to this later.
                Last edited by getnamo; 03-12-2019, 07:28 PM.
                Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                Comment


                  #23
                  I do not understand this type of error?
                  Attached Files

                  Comment


                    #24
                    Originally posted by Jmeletou2014 View Post
                    I do not understand this type of error?
                    python is space sensitive, you have to either use tabs or spaces consistently. Check your file in something like sublime text to see what spacing you're using.
                    Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                    Comment


                      #25
                      Originally posted by getnamo View Post

                      https://github.com/getnamo/tensorflow-native-ue4 is now functional if you want to use the tensorflow c_api directly, but there are no convenience unreal type bindings yet. Will come back to this later.
                      Thanks! I take it this means this plugin could be used for inference on tensorflow networks that have been built outside of Unreal, assuming we can find a way to package the .pb files? I imagine there'd still need to be some work to get tensorflow itself to compile on consoles as well, but this is a good start. Thanks!

                      Comment


                        #26
                        Originally posted by David.Bromberg View Post

                        Thanks! I take it this means this plugin could be used for inference on tensorflow networks that have been built outside of Unreal, assuming we can find a way to package the .pb files? I imagine there'd still need to be some work to get tensorflow itself to compile on consoles as well, but this is a good start. Thanks!
                        That's the idea, here is some example c_api for loading a graph and running a session: https://github.com/Neargye/hello_tf_...ession_run.cpp. Wrapping that into a more unreal way would be the next step to make things easier.

                        In terms of packaging the .pb files, it's not a big issue you can just add your pbs as runtime dependencies and they'll get packaged along with other things, see https://github.com/getnamo/tensorflo...w.Build.cs#L32 on how that's done for the script files.

                        Keep in mind that another way people do machine learning is to run it as a cloud service and just pipe data to your server and get results back, then you don't have to worry about compatibility for your platform of choice.
                        Last edited by getnamo; 03-15-2019, 03:51 AM.
                        Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                        Comment


                          #27
                          How does the package scaling version work?

                          Suppose I package a version and I use CUDA and cudNN but the PC it would be running on doesn't have CUDA or cudNN but has a card that supports them, would it still work?
                          How would it work on a system not having a GPU can it detect and run the processor version?

                          Comment


                            #28
                            Originally posted by envenger View Post
                            How does the package scaling version work?

                            Suppose I package a version and I use CUDA and cudNN but the PC it would be running on doesn't have CUDA or cudNN but has a card that supports them, would it still work?
                            How would it work on a system not having a GPU can it detect and run the processor version?
                            See https://github.com/getnamo/tensorflow-ue4/issues/39 for a discussion about this problem. The plugin currently assumes you distribute the correct version by specifying it in the upymodule.json. There could be some ways of detecting the compute capability of the computer it runs on and selectively pull dependencies via pip and that enhancement would be a good contribution to the plugin. For now I'd recommend using the cpu version if you're unsure of environment.

                            Alternatively if you do have a model that is more inference focused, there may be mileage from using the tensorflow c_api to run on say a frozen model or .pb file, there are native dll distributions available that come with the requisite gpu binds already directly embedded in a tensorflow.dll. In https://github.com/getnamo/tensorflow-native-ue4 I'm using those dlls and exploring how a more native tensorflow plugin might look. In it's current state, it will correctly load the dll and you can call the c_api from c++, but there is currently no example or unreal specific helping code (Still very WIP). I suspect this approach should be more amenable to being embedded in games with far fewer dependencies, while letting you train/research using e.g. the python based tensorflow-ue4 plugin.

                            Finally another common approach is to run the tensorflow code on a server/cloud instance and to just use networking (e.g. the socket.io plugin) to pass data back and forth. That comes with it's own drawbacks though (requiring internet connection, scaling costs).
                            Last edited by getnamo; 03-26-2019, 12:30 AM.
                            Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                            Comment


                              #29
                              Another simple question although this is more related to the python plugin but I didn't know where to ask the question.
                              https://github.com/20tab/UnrealEnginePython/issues/697

                              I have a py file inside a folder inside scripts, or any folder inside content folder.
                              Then I add this path to additonal path under python but when making a pyactor and adding that class to python module, I am unable to make it work.

                              Any idea how to make something like that work?

                              Comment


                                #30
                                I am getting the following error when opening a component after installing tensorflow 2.0
                                Click image for larger version

Name:	Capture.PNG
Views:	50
Size:	167.2 KB
ID:	1605275

                                Comment

                                Working...
                                X