Bringing Deep Learning to Unreal Engine 5

With the release of UE5, Epic finally integrated a native plugin to directly run machine learning models directly in engine. I’ve personally been interested in this plugin from the very beginning, but, even after several months from the release, online resources are limited if barely existing at all.
So, after spending some time trying to collect information and figure out how to use the NeuralNetworkInference plugin, I decided to write a series of articles to share my knowledge.

This is part 1 of this series where I will cover how to integrate the NNI plugin in a blank project and how to use it: Bringing Deep Learning to Unreal Engine 5 — Pt. 1 | by Weird Frames | Aug, 2022 | Medium

2 Likes

Hi gvecchio95!!

I am also struggling to find any useful info about this new NNI plugin. Do you have any idea when Part 2 might come?

Thanks a lot for starting this series, it’s a very interesting topic :smile:

Hey, @TheSilver812!

Part 2 is almost finished (took some more time than Part 1). Hopefully, I’ll be able to publish it by later today/tomorrow.

I’ll post an update when the article is online. You can also follow us on Instagram as we will be posting more updates there.

Thanks for the interest in the topic. :grin:

1 Like

Part 2. of the series is online! :tada:
It was really hard to write it in a way that was detailed enough without being boring and messy, hope you’ll like it.

In this installment we will go a little bit deeper into the implementation details of the real-time style transfer approach. Most concepts and techniques can be easily applied to different tasks involving neural networks in Unreal.

Bringing Deep Learning to Unreal Engine 5 — Pt. 2 | by Weird Frames | Sep, 2022 | Medium

1 Like

Really nice few articles, when is Part 3 coming? :eyes:

I’ve been working on integrating a dynamic weather system into my game, and I’ve followed numerous online tutorials and documentation to the letter. However, despite my best efforts, I’ve hit a roadblock – the weather effects, particularly rain and snow, seem to be displaying in the wrong direction within my game world. It’s almost as if the rain is falling upwards! I’ve meticulously checked my settings and blueprint logic, but I can’t seem to pinpoint the source of this unusual behavior. :slightly_smiling_face: