Hey, so I think the problem you are having is because you are using a video in the media plate. If the video is not playing, it will just appear black. I had a similar problem when using a webcam as a source, it has to be started up for it to appear.
You got it to start playing when you had the media player window open, but when you close the window it stops playing so your “media plate” goes black. Open up your media player window and get it to start playing in a loop there, then leave that window open and go back and look at your media plate, you should see the video playing there now. I’m not sure if there is a way to make it keep playing when you close the media player window (there probably is, I just don’t know it). To get it to play when you actually run the game, you need to put something in the level blueprint to start it playing.
The reason some things are not all that straightforward is because this is a game engine and not a video editor, a game might have tons of different videos in it and they can’t be playing all the time, even in the editor, something has to tell them to play.
By the way, if you get the weird green corners like you see in some of the composure samples, this is from devignetting in the chroma keyer. This is for correcting for cameras where the outside of the image is slightly dimmer than the center. I think if you set the first devignetting value to 1 and the others to 0 it will look ok, you may not have to do this, I’m not sure what the defaults are.
I went ahead and setup the chroma-key and it looked pretty good, there is a bit of a green edge around your character, probably because whatever you used to make the animation was doing some smoothing of the edges or maybe just because the video is compressed. The quickest solution for this is to mess with “erode” in the keyer which will just bring in the edges of the key a little bit. Eventually though, you can probably setup your animation program to output a video with an alpha channel and render your character with a black background instead of a green one. I haven’t played with alpha channels in video much, but if your animation program can generate one, composure can use it INSTEAD of chroma keying and the results should be better.
its not the ideal solution but i think I’m getting somewhere. I was able to use a PNG sequence as footage which made working with the alpha channel a bit easier. I still get artifacts around the edges. If i can figure out a way to keep the dialogues and audios synchronized it might be ok. Wish i could do it with video, I feel that keeping the sounds synced within the video would be easier and provably less processor hungry. Experiments will continue.
Been able to set up basic cycles using flipbooks and basic crowds using flipbook and planter… results are decent but need to figure out how to have the flipbooks start at different frames to make them look out of sync.
But in terms of video. I was able to figure out how to import webm video which supports alpha channel. There’s a setting in the player that allows you to select the decoder. So if you enable the webm player in plugins, the player gets a additional webm decoder under its options section. But unreal craps out if your webm doesn’t has a sound layer. So its kinds useless unless your webm has audio. There’s another player called wmfPlayer… that one seems to also be able to play webm, with or without sound.
Of course now I have to come up with a streamlined solution for exporting my webm videos if I’m to work with that format. So i revisited the whole chromakey thing. I was able to get somewhere with it but its still not perfect, but i can see that with some tweaks I may be able to get somewhere. I’m thinking the chromakey route might be the better solution, since i don’t have to preprocess any webm files.
Oh, also i figured out how to use PNG sequences as video, but seems like it bogs down the system whene ever you have a few hundred frames, it give prefect transparency but also audio is not keep synchronized.
Glad to hear you are making progress. If you are getting artifacts around the edges of your key it could be due to video compression, almost any kind of compression can generate artifacts around hard edges, this could cause flickering at the edges of the key. In my stuff I’m working with uncompressed live video. If you are using compressed video, turning the quality all the way up might help reduce artifacts around the edges. I think they just added ProRes codecs to Unreal, but it might only be for output and not input of video.
Also something to think of, in your alpha channel video, what’s the RGB color in the area with transparent alpha? If it’s a strong color it will bleed through if there is any compression artifacts at the edges of your alpha channel. You might be able to reduce this by making the color of the transparent area something neutral (ie, not black, white or green) like a grey or whatever the predominant color of your 3d background is. So if there are any errors in the alpha channel the color bleeding through won’t be as noticeable.
I think i remember reading somewhere that EXR works better for media sequences, I had trouble with img sequences in the past, so I never used it for production
also, toontitan, you could also try using 2 video clips, one with the color, one with black and white for the alpha maybe, not sure how that would turn out in video memory usage though
Just checking in to see if any one else has made any progress in this.
On my side i still have not been able to crack it. I been able to create flipbooks out of png sequences which seems to work great but with resolution limitations. Also not syre how to synchronize sound to them.
The idea solution is transparent video with embedded audio source. Has anyone else done any experiments on this?
Hi Toontitan, Glad to see a fellow iClone user in here… I happen to have some degree of success with the “chroma-keying” part of what youre trying to do. The main difference with what I’m doing is that I am using a live feed versus importing in a video… but the concept is pretty much the same whether you use a pre-rendered video, or live footage from a camera. The main part of this whole idea of chroma-keying inside of Unreal Engine is where you will run into so many different and sometimes complex ways to remove the green. I have watched countless videos on Composure and just couldn’t get things to work properly – and it’s mainly to do with my own inexperience with UE4.
When I figured out a solution to my first issue, which took me forever, I quickly made a video about it to help others who might be trying to do the same thing. My solution was to use OBS to do the keying, which mostly worked, but I wasn’t very happy with that workflow and the results were only “OK” – The simplicity of using OBS came with some drawbacks like lots of noise on the main footage, and some fringing or artifacting around the edges and also green-spill.
This week, I started to do some more experimenting… trying to cut OBS out of the workflow, and I think I finally figured it out… It’s kind of difficult to write a step by step here, but I plan on making another video as an update to my first video (which had some flaws since I literally put the video together as fast as I could in a day) – but works great if you don’t mind using OBS as the middle-man. Maybe some of the concepts in the video will give you some more ideas. But like I said, I have a new solution that uses a simpler workflow and I’ll share how to do it soon. Here’s a link to the first video if you’re interested:
Thank you so much for this! I learned so much, I think this will come in handy for doing live animated character via OBS! Thank you for turning me onto the NDI thing. its awesome.
Im gonna post a followup later today with how I finally managed to do chromakey. I still don’t know how to keep sound in my videos but that’s another hurdle i hope to figure out soon.
Here is my results so far. It does yield great chromakey. Only thing I need now is to somehow be able to bring in and use the audio. There was some old videos that show how UE used to bring in a separate audio file that you can re-sync, but apparently that was several versions ago. Also i can’t seem to be able to figure out how to have by object cast a shadow. It works fine with image sequences, cause i guess they have some sort of UV thing but there’s gotta be a way to do it with video as well.
How To Make A Video Chromakey Material in Unreal Engine
