Ultimatte 11 or 12 and Unreal Engine...for film... expert help needed

Development Cinematics & Media
1

Hi:

Well, going further down my virtual studio path…

I have a Blackmagic Design Ultimatte 12 (although now I find out that the Ultimatte 11 is as good and very cheap here in the USA- less than $3,000!).

I need to create a virtual studio, and I was planning on bringing in SDI live video (people on green screen) into Unreal- do green screen removal and comp work using composure.

It works, but only OK…

Ultimatte 12 (11):

The reason:

  1. It does incredible work in terms of compositing- extremely good green screen removal, and a ton of ways to integrate your foreground, background, etc. to make it all very realistic- much better, I feel, than the ability of Unreal, mainly in the area of the removal and ‘fixing’ of the green screen process. I always feel that the green screen work inside of Unreal results in edges, etc. that make the image look a bit ‘pasted on’, (no matter how hard you try for realism).

  2. It seems that the ‘big boys’ are doing something like this, using the Ultimatte, I assume, for all the same reasons.

  3. You have basically, two ways of approaching this, as far as I can tell:

a) Feed Ultimatte into Unreal, have Unreal do the comp work, not the green screen removal- use Ultimatte for that. Problem is, now that I dig deeper:

Ultimatte outputs two required SDI signals: one is the key (your live scene) and the other is the matte. So, you need to input two SDI streams into Unreal, and have Unreal combine them (key+matte) so you end up with the transparent background (hence, the required matte output). The key output has no alpha channel, so the background is black- you need the combo of the key and matte.

I guess, inside Unreal, you create two media profiles to accept 2 SDI channels (don’t know how to do this), and then somehow combine them inside of Unreal…don’t know how to do this either.

Above comments are the way I’d like to do it- help…!

The other way, which gets you some more functionality, is use Ultimatte as intended- Take your scene in Unreal and output it over SDI to the Ultimatte background input. Live video (camera) into Ultimatte- and, it does all the comp work.

Not as flexible as doing it all in Unreal (??). And, apparently, the ‘big boys’ do it inside unreal (Ultimatte into Unreal).

I do see you can create a media output (Unreal to SDI) and you can configure the output to be the key, or key+matte.

Anybody have ideas or knowledge as to how to accomplish this?

Thanks!

2

Hey
Need to to the same thing .
I have 4 Ultimattes 12 and i need to understand the workflow with unreal for acquire the Fill and Key with composure i think thats the way …
Did you find some ways on it ?

1 Like
3

Same EXACT question… Any ideas?

4

Hello， have you found the feasible solution for this Question？

1 Like
5

Hello , et al’…did anybody find a solution to the questions asked on how to.use the Ultimatte 12(11) in Unreal Engine using 2 SDI?

6

This could be useful tutorial

7

Im trying to do this exact same thing – take an amazing key from Ultimatte 12 and bring it into Unreal without destroying the great key. While I have yet to be able to solve this I am working on a solution. I did find a couple things:

  1. VComposure – not sure how this would work with Ultimatte exactly, but it is able to use a soft alpha matte to retain the transparency of things like hair, etc. See the website: https://pixelstorm.uk/ They are working on a final v1.0 release that I think will be ready in the coming days.

  2. See Off World Live’s video about using the key and fill from the Ultimatte 12 with Unreall Engine. https://www.youtube.com/watch?v=BOFvQ8VshQ0

Personally, I’m trying to find a way to get the keyed plate (keyed with Ultimatte 12) into Unreal as if it were a MediaPlate, or a 2D plane in 3D space. I’m not sure if I can do that using Composure as they always talk about creatting a matte for foreground objects. Maybe this is necessary for live, but if Im just rendering from Unreal and baking in the green screen footage then I dont see why I couldn’t just do my camera movement in UE and use the Composure with Ultimatte in lieu of a MediaPlate with the same results? I’m still trying to figure this out myself.

8

I am building a studio with the exact same pathway. I am still far too new to Unreal to offer an in-engine opinion however there is another way to do this. I have been concerned with being able to grade footage afterwards in post and my “external” pathway option is this:

Camera feed, with camera tracking, into Ultimatte 12. Pipe the Unreal background into the Ultimatte 12 background feed and compose it that way. Pretty traditional. However I use Hyperdecks to capture the Background, Matte, and Program feed for each camera. I then take the background, matte and original camera footage into Resolve and am able to grade both the background and the key any way I want to. I use the recorded matte and the camera “key” footage and this works perfectly.

Interested in everyone’s thoughts regarding this process. And yes, over the next few months, as I continue to learn Unreal, I will be working on how to being both key and matte into Unreal.

2 Likes
9

Interesting, I am using the same process. But have to ask you a few questions about color space. I want to share my setup with you. Are you on Instagram or anything?

1 Like
10

Happy to answer anything I can. I’m not on instagram however.

11

So I’m basically going to be compositing in the ultimatte as well. Previously I was trying to do the key and feel in this software called assimilate live FX but I was having trouble exporting the alpha and I got immediate results from the ultimatte.

So currently my workflow is the same as yours. Except I don’t have the hyperdecks. The output of the program from the ultimatte is being recorded in my live FX software that does all of the dailies available for immediate review. What color space is coming out of unreal if I’m using media capture to output over SDI? Have you been able to grade the rec 709 that comes out of the ultimate and have it hold up in pretty good quality?

12

Wertup123, Here is where I have taken a turn on the traditional ways of managing the Ultimatte footage. Here are the four pieces of footage I have, all camera tracked:

Unreal background - Hyperdeck 1 Recording
Matte Output from Ultimatte - Hyperdeck 2 recording
Key footage from Red Camera - Uncompressed on camera media
Program Footage - Unreal background, and Matte and Key from Ultimatte output - Hyperdeck 3 recording

Using this method I have two options.

Option A: Take program footage from Hyperdeck 3 recording and bring that into Resolve as a Rec 709 colorspace. Grade form there.

Option B: Bring the Matte, Unreal Background ( rendered or Hyperdeck 1 recording ), and Red camera uncompressed footage into Resolve and using the timecode on each piece of footage match it all up. Grade the Red camera footage using Red IPP2 workflows. The matte will key the uncompressed footage and it will all lay over the Unreal background.

There is an Option C however I don’t know it yet. I just had my virtual computer arrive today and for the next month I will be learning Unreal. My thoughts after doing some research will be to bring the Key and Matte from the Ultimatte’s into Unreal and grade everything there. Still lots of learning to do in this area and honestly I don’t know what I don’t know.

Hope this helps and was what you are looking for.

13

As of right now for me I originally set out to do option one. But now for the past two weeks I’ve been doing option two. It’s a much more streamlined process and I highly recommend you look into assimilate live FX. Check out their website and you would see how you can preview dailies and have all of your takes live on set after you finish shooting them. It’s what the pros are using to do virtual production and or traditional Hollywood when having to deal with reviewing footage. With that being said, which tracking system are you using and which PC did you go with for unreal? I’m about to start my PC bill dedicated just to running unreal engine.

14

I haven’t tried the workflow of bringing the ultimate into unreal engine. I’ve been trying to avoid composure as I do not want the CPU to be bogged down with inputs coming in. Plus using a blueprint to combine the fill and key just slows me down. Those are my thoughts on it.

15

Again, still early stages for me. I have the Vive Mars CamTrack and two FizTracks sitting on the shelf ready to be installed in a few weeks. Regarding the computer build, I just received that yesterday. I ended up getting the below listed system. To me the motherboard was important to manage my PCI lanes. I also upgraded the power supply so if I were to get an RTX 6000 Ada Generation card for use with a volume, it really is just plug and play.

As I have said before my next few weeks will be learning the very basics of Unreal so I can start doing some very simplistic output.

Computer Specs
Threadripper Pro 5995WX with a 360mm Liquid cooler
WRX80 w/ WIFI+BT+10G motherboard
8x16GB DDR4-3200 ECC
Drive 1 - 1TB Gen4 NVME
Drive 2 - 4TB Gen4 NVME
MSI 4090 Gaming X Trio 4090 24GB GPU Air cooled
Silverstone RM51 5U case
Dark Power Pro 13 1600W Titanium power supply
2 Decklink 8K Pro cards

Total cost was $16,800

1 Like