Hey folks,I decided to share my almost 1 year journey of developing a video recording system for Unreal Engine 5.
If you want to read full story here is the link: https://peterleontev.com/blog/unreal_video_encoding/
You can buy my plugin on Marketplace: https://www.unrealengine.com/marketplace/en-US/product/runtime-video-recorder
I learned quite a few things:
- If you are trying to implement even a basic video recording system (i.e. to produce .MP4) then you should know that Video encoding industry is so messed up these days. (licenses, patents, performance, etc.). There is no good way, only lesser evil.
- x264 is a king among all software-based codecs. I used OpenH264, SVT-AV1 and some other implementations but x264 rules them all out.
- Hardware accelerated video encoding on Android is sophisticated. Well, it is working but it is so nuanced and non-transparent wrt implementation details i.e. input frame pixel format and/or even resolution. Main issue is that hardware vendors decide HOW to implement codec support and that often leads to non obvious input requirements especially memory layouts.
- For curious souls, Unreal GameplayMediaEncoder system can be your friend if you don’t want to use any thirdparties. It even supports Audio. However, keep in mind that solution is only available on Windows platform.
I managed to find a way to implement a basic video recording system for Unreal 5. That means two things:
- You can integrate it into your app/game and use it on any major platform (except for Mac)
- You can easily add your own codec implementation i.e. if you want to use some other container format (i.e. vp9) or even encoding layer (i.e. nvenc). Just be mindful of software licenses!
I’m proud about the following:
- Audio support
- I found codecs/implementations with compatible licenses (for commercial purpose)
- I am also satisfied with performance on Windows, Linux and Android (including Oculus VR). Windows implementation uses Hardware acceleration (almost all vendors are supported → Nvidia, AMD, Intel. Linux implementation is software based only for now (I will likely end up integrating libva!).
- I’ve added support to record Render Target and make a viewport screenshot in async way.
Enjoy!
