Im wondering is it possible to get effects similar to those shown in picture below and use it in a game in real time. How to code something like that? Is it necessary to use neural networks? Would that slow down the performance too much?
Hi, running deep dream will be heavy on performance since you need to forward and backward pass the image several times, so will most likely not run in realtime (depends on what network).
The image you have shown does not look like deep dream, but like style transfer. Style transfer is much lighter on performance, since you only need a single forward pass through the network. No idea whether that will run in realtime though, maybe with a lot of optimizations.
The next problem you will have, is to keep it consistent between frames.
Here seems like someone used a neural network to do style transfer in UE GitHub - UnrealXinda/NNPP: A UE4 Plugin of Neural Network Style Transfer for Post Processing is on lower resolution though but seems to run in real time.
So if you want it to run in real time, I guess you need some way to do it with some shader magic without neural networks…