Live action footage projected onto 3D widget inside game

Development Programming & Scripting UI
1

Projectedwoman.jpg
Projectedwoman.jpg1172×702 165 KB
Hi guys,

I played around with projecting some hud like graphics onto a 3D widget and
connected that to an actor I placed in a test level

?v=sOb4QdCO4DA

Here is an example of a movie shot where I used projections + alpha channel
onto cards in 3D space (Nuke) to remove cars from a shot:

?v=LBfQNKXugA0

It works fine but I want to know if I can project live action footage onto a 3D widget like this.
In other words if I want the avatar to walk up to an actual movie being projected in his world.
Maybe stand there and watch a movie.

I know I can add a texture or image on this but how about if I want a movie clip?
Also can I import a movie clip with an alpha channel so that there can be a “cut out” of a person
projected?

Thank you.

2

O.k. so I figured out part of this myself. Took a green screen shot of woman into Photoshop, keyed out green, saved out “for web” as 8 bit PNG file with transparency dithered selected. That seems to work fine. Now I just wanna see if I can do this with a movie file.

3

O.k. so doesn’t look like I can bring a movie file format in as an image or texture.
PNG works so maybe I can flipbook them ??
It would be nice if we could bring a SWF or FLV or GIF in with alpha channel.

4

Take a look at media texture assets -> https://docs.unrealengine/latest/INT/Engine/MediaFramework/index.html#mediaassettypes&importingmedia But I dont know if you can add an alpha channel to that.
Otherwise use a flipbook material :slight_smile:

5

O.k. thank you .

6

Yeah it would be nice to somehow be able to hook an alpha channel up to this media texture / projection.
You can multiply with alphas in U4 so maybe there is a way to do that.
You just have to render the alpha out as a duplicate of the movie but in black and white.
White masking the area you wanna keep and the rest falls away.
Just to get away from ending up with a square box projection :slight_smile:
Same deal with the buttons in the UMG.
Who wants a square button? :slight_smile:
If I have the time I will dive into this and see if I can cook something up.
I will share.

7

Here is an interesting one on static textures.
This is the idea but to do it with media textures …

?v=aelOH9jjt2g

8

Just checked and a media texture made into a material works just fine.
And it does have a red green blue an alpha output / function - so looks like its possible.
I just have to see what media file I can create that has an alpha channel in it I guess
or create a 2nd media file that has the animated mask as black and white and then channel swap.

9

Make sure to take a look at this thread: https://answers.unrealengine/questions/116661/can-45-video-textures-support-alpha-channels.html :slight_smile:

10

Great pointer! Thank you so much. That is exactly what I was looking for.

11

Here is progress.
It kinda works but I am having problems syncing up the two media textures so they play + loop on the same timing.

?v=PsFohsHADxU&feature=youtu.be

TwoMediaTextures_with_one_as_alpha_mask.jpg
TwoMediaTextures_with_one_as_alpha_mask.jpg1054×622 123 KB

12

AHA! Figured it out !!! You ctrl+select both media player icons in tray, right click on one of them (both selected) and hit “play”
Both play in sync !!!
Hehehehehe. :slight_smile:

13

?v=jznu-3qOPgU&feature=youtu.be

Here is the result.
Its still out of sync though but it looks like you can hook these media players up to a master controller. ??
There is also lerping … if I can figure out what media movie file U4 can take that can support an alpha channel …

14

The other thing to do too, just as a quick solution, would be to color correct the clip, pump up the red channel so it would
affect the green and blue. Then to take the red channel output and hook that into the mask input in the material. Then add a color correct node between the “base” output and input of the material that takes all the red back out again. That might work. So you would use the red channel as an alpha or mask value then.

15

That looks interesting. Thanks for posting your solutions and work arounds, it’ll help people who search the forums with a similar problem.

Now, that gif is interesting. I have to ask, what type of game/project are you working on? :rolleyes:

16

Hi actually no game yet. I am just learning U4 coming over from U3. I have project in mind but there is a lot of testing and development going on and I will keep things under wraps until I have a trailer or something to show. :slight_smile:

Thanks for reading though. Yeah any creative project, film or game is always all about workarounds :slight_smile:

17

The media file in this test is a .mov (QuickTime), 24 fps (the original mp4 was 25 fps) with H.246 codec and bit rate of 5000 KB. No fields. sRGB color space. The original clip of the girl has her against a green screen. I keyed the green screen out to create the alpha with Keylight inside Nuke, then channel swapped to create a black and white RGB which I rendered out as a 2nd QuickTime with same specs as the RGB one.

18

The reason for this test was because I want to create complex motion graphics that are not restricted to “button” shapes as in the UMG at the moment. So I would create / composite my motion graphics in After Effects and just render everything out with black and white mattes and build a hud . menu system this way. However I dont know how this will slow the game FPS down though. But what I had in mind was this from Dishonored where they used Scaleform:

?v=1yLldqE3b4U

19

52bc3afa94739eb4bc987e4bd4d0e909e57430a7.jpeg Here is how I pulled the green screen and swapped channels in Nuke.

20

Here is an interesting effect I got by playing around with hue shifts, contrast rgb and Scalar factors inside
Unreal 4:

?v=aiPE–bgl5o&feature=youtu.be