TensorFlow

[getnamo]

Machine Learning Remote Plugin (remote variant of tensorflow-ue4)** Released.**

The remote variant of the plugin is now functional, check out: GitHub - getnamo/machine-learning-remote-ue4: Machine Learning plugin for the Unreal Engine, encapsula for the plugin and GitHub - getnamo/ml-remote-server: Server component of https://github.com/getnamo/machine-learning-rem for the complement server. The plugin should be usable in a much wider variety of platforms (anywhere the socket.io plugin can be used so: win, linux, mac, android, ios) and you can host your server part anywhere that supports python 3 (typically linux, win, mac).

If you wanted a lighter client or a more flexible backend this should be the one to check out. Let me know if you use it and if you’re missing some API.

[getnamo]

@TechLord

This is awesome. I want to train different with AIs with different Game Lore/World Datasets to drive NPC Chatbots. I’m open to any conversation on this topic.

You would probably be interested in models like gpt-2 Better language models and their implications which now has the full 1.5B parameter model released: GPT-2: 1.5B release (it’s super heavy though). This can predict fairly decently additional text based on a snippet of input text. It can even answer some questions about a text (reading comprehension).

Below was an example I used for testing reading comprehension. Everything below SAMPLE 1 is generated by gpt-2 in response to the text above it.

https://pbs.twimg.com/media/EIuZ9eGUcAEyuHo?format=png

[TechLord]

@getnamo Thanks for your insight.

@getnamo

You would probably be interested in models like gpt-2 Better language models and their implications which now has the full 1.5B parameter model released: GPT-2: 1.5B release (it’s super heavy though). This can predict fairly decently additional text based on a snippet of input text. It can even answer some questions about a text (reading comprehension).

Below was an example I used for testing reading comprehension. Everything below SAMPLE 1 is generated by gpt-2 in response to the text above it.

https://pbs.twimg.com/media/EIuZ9eGUcAEyuHo?format=png

[getnamo]

I’m happy to announce that the Tensorflow plugin was selected as a MegaGrants recipient: Epic MegaGrants reaches $13 million milestone in 2019 - Unreal Engine

This should help make the multi-variant component refactor (tracked at Refactor to support Native and Remote variants · Issue #53 · getnamo/tensorflow-ue4 · GitHub) a reality. The remote variant has already been implemented and released, we now just have the native variant currently outstanding. Keep an eye out on the issue to track progress for this upgrade. Let me know if there is some specific API you’d be interested in having in either case.

[Ben_Cykyria]

That’s great! Congratulations!

I’m looking forward to the native plugin. Just getting into ML. It’s a tough journey.:rolleyes:

[envenger]

With the python plugin being dead, what’s the plan next to upgrade this.

[FAHR]

How did you manage to upload those gifs?

[getnamo]

@FAHR

How did you manage to upload those gifs?

https://i.imgur.com/Roqll8Q.png

Insert image, paste url to e.g. imgur and do not retrieve locally.

@envenger

With the python plugin being dead, what’s the plan next to upgrade this.

I haven’t had a chance to look at the UnrealEnginePython breaking changes for 4.24 yet, but will likely look into making the split between GitHub - getnamo/MachineLearningRemote-Unreal: Machine Learning plugin for the Unreal Engine, encapsulating calls to remote python servers running e.g. Tensorflow/Pytorch. which would still use your local python but via socket.io transparently and GitHub - getnamo/TensorFlowNative-Unreal: Tensorflow Plugin for Unreal Engine using C API for inference focus. for native/inference focus.

The remote is ready to use on e.g. windows systems, I need to squash a bug or two to make it run well on linux. The inference focused plugin needs BP api work.

[gustmes]

Hey,

I tried using the plugin for unreal version 4.23, the project loads up to 71% and then disappears. When I do a debug build from Visual Studio, I get the following error:

Fatal Python error: Py_Initialize: unable to load the file system codec

I have python 3.8 installed on my machine, do you know what is the source of this error or what can I do to make the plugin work?
Thanks

[vegdrop]

Hey,

I’m starting working with tensorflow plugin. I would like to use neural style transfer techniques.
My question is : Is there currently a way to input an image rather than a json?

[getnamo]

@vegdrop

Hey,

I’m starting working with tensorflow plugin. I would like to use neural style transfer techniques.
My question is : Is there currently a way to input an image rather than a json?

Current pipeline uses json for transport. If you input an image it will convert into a Json array value of floats which can easily be loaded into numpy.

1. a binary serialized image is an enhancement has been on the docket for a while: float array pointer pass · Issue #7 · getnamo/TensorFlow-Unreal · GitHub. Likely with the push toward tcp oriented data pipe, just serializing this instead of pointer pass is more likely.
2. ideally we’d have a way to mark a texture as a shareable and then be able to re-use this pointer in tensorflow, I’m unsure if tensorflow has ops that supports this in theory.

If you come up with a better solutions, code contributions are welcome!

[Analysierer]

Will it work with UE5?

[anonymous_user_d68c455a]

I am a newer on unreal engine.
there is one problem in unreal engine as following, is there some way to use python and tensorflow on the runing mode??

---

Scripting the Editor using Python

Unlike Blueprints, the Python environment is only available in the Unreal Editor , not when your Project is running in the Unreal Engine in any mode, including Play In Editor, Standalone Game, cooked executable, etc. That means that you can use Python freely for scripting and automating the Editor or building asset production pipelines, but you cannot currently use it as a gameplay scripting language.

[Alimohammadlog]

how to launch this on ue5.3