What video codec for HiRES video with Alpha Layer?

We have an Unreal environment running on Windows and we need to use hires video (up to 4K)
What codec is best used?

The video we want to play inside the engine has Alpha layers.
For performance reasons we prefer to use a single file, but the video file formats that have an Alpha layer built in are limited. We can circumvent this by using a second video with an alpha channel.
Either way, what video gives us the best performance?

H.264 mp4 gives us the resolution, but does not provide 100% smooth video.
Any tips?