It’s been a while since I’ve posted here, I took a break from game dev after making my last game and dealing with all the grief that it entails. However I have not taken a break from the Unreal engine. One of my favorite music artist “Grimes” put up some green screen footage and asked people to make something cool with it and I seized the opportunity. I wanted to see if I could integrate the footage into the engine and render it out. Why you may be asking? Well I’ve made static virtual sets before on the tricaster at my last job but I always wanted to do something more dynamic. Some of the benefits are have the subject’s light and presence affect the scene by casting shadows and appearing in the reflections. I though it would be easy but it kind of was not.
I looked at the Unreal Engine chroma keyer and I decided I did not want to use it for reasons, so instead I keyed it out (badly…) the video in resolve and exported a mask.
The first problem was the masked opacity shader is only 2 bits so it gave horrible edges. The next option was the translucent material, while giving smoother result had it’s own bag of troubles, for example not casting shadows, or receiving shadows or appearing in reflections which totally defeated the purpose of doing it in the unreal engine. None the less I found out it was possible to make the translucent shader cast masked shadows so at least there was that for now. For the reflections I cloned the image plane and put a masked version just behind it which actually worked surprisingly well. Next problem was the DOF for all the translucent shaders, it tended to stick out like a sore thumb however this was easily fixed by not rendering them in a separate pass, thankful this option is available in the project settings. The only thing I wasn’t able to figure out was how to get it to receive shadows…
The next problem was the exporting, I love sequencer, it’s pretty great but I had trouble during the final renders, the mask and the color tracks would go out of sync randomly on some frames, like one could not keep up with the other. I tried doing the final render in 4k but the sync issues were too much so in the end I settled for 1080.
I used the Paragon Slum level so credit to the great artist at Epic games who made it.
Without further ado here is the video:
If you would like to try it out for yourself, here is the green screen footage:
In conclusion I think this is a great process for quickly doing videos like this, the big benefit is not having to wait hours for renders to finish. I was able to iterate many times because exporting the frames were so quick!
If I may suggest a couple things to make this type of stuff easier in the future; A “cinematic translucent shader” would be cool, it would act like the masked opacity shader except be 8 or 10bit so there can be smoothness, shadows and reflections. Since it would be for final renders it doesn’t matter if it’s not usable for “realtime”. The second thing would be to enforce the “sync” in the sequencer video playback for the color and mask image sequences.
Thank you for taking the time to check this out. If you have any questions or comments feel free to ask!