Hi there - loving the power of Sequencer for film work here, especially postviz workflows using Greenscreen chromakeyed video on 2D planes, with Sequencer and the Cine Cam marketplace app for a VCam.
Trying to make this work with audio however is currently running into some road blocks though. We’re having issues where the audio won’t play back from HAPQ media files (which seem to be the best for performant chromakeying source media).
This seems confirmed here ‘As of 4.23, HAP playback does not support embedded audio or timecodes.’
The things we tried were 1) ProRes codec and while it’s perhaps optimal quality for chroma keying, the playback peformance in Unreal (CPU usage) is far worse than HAPQ, and not viable with multiple layers/streams playing at once in a scene. And 2) we tried with h264 mp4 files. Performance is acceptable, however the chromakeying is poor due to video compression (but at least the audio works when you set audio to a Media Player via the Sequencer)… This is our workflow for now, but the chromakeying is compromised.
So hoping Unreal could give this workflow a bit of love and add audio playback support to the high performance HAPQ media file format - for playback on Sequencer! This would be a massive improvement to the workflow for film post-viz and in-engine production using chroma-keyed live-action filmed footage.
The other nice helper for workflow would be if a MediaFile’s audio would automatically play back in Sequencer, without having to use a separate Media Player to play the audio alongside the video (which we have on a Texture, and a chromakeyed material instance landing on a 2D plane), as to get that to work currently you have to keep the Media Player open (otherwise we don’t hear audio). But that may be a workflow mistake here!
Anyhow it’s pretty exciting what you can do already with this, so here’s to the next iterations!
