Announcement

Collapse
No announcement yet.

Realistic Face Markerless Motion Capture

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Realistic Face Markerless Motion Capture

    Hello Unreal developers! I´m developing an in-game cinematic trailer for my own IP, so I created this test that has nothing to do with the story of the project, but a lot with the quality I´m trying to achieve! This is markerless motion-capture, then rendered inside Unreal Engine 4. If you want to know more about my future projects, feel free to add/follow me at: https://www.facebook.com/BraVlio.FG or https://www.instagram.com/brav_fg/

    I want to thank David Marmor for the base model and blenshapes, and Baongoc Vu for letting me use and tweak his eye material. I was in charge of making the UVs, all the texturing, materials, secondary blenshapes, re-rigging the whole thing for game engine, lighting, mocap setup and animation. One of the main goals while planning this test was not only to show quality, but to know how the engine would manage to handle a complex model. To see if a real time solution could compete with pre-render quality. I was surprised that the engine handles it pretty well, running at 50-60fps at 1080p, and bare in mind that is a very high poly model because is a test intended for cinematic quality, also have complex shaders and some textures are even 8k!



    Click image for larger version  Name:	FloatingHead.jpg Views:	1 Size:	493.1 KB ID:	1226318

    Click image for larger version  Name:	Front_Screenshot.jpg Views:	1 Size:	373.9 KB ID:	1227037

    Click image for larger version  Name:	Eyes_Screenshot.jpg Views:	1 Size:	372.3 KB ID:	1226319

    Click image for larger version  Name:	Nose_Screenshot.jpg Views:	1 Size:	274.5 KB ID:	1226320

    Click image for larger version  Name:	Mouth_Screenshot.jpg Views:	1 Size:	363.9 KB ID:	1226321

    Click image for larger version  Name:	HairTest_03.jpg Views:	1 Size:	319.4 KB ID:	1226322
    Last edited by BraV FG; 01-05-2018, 06:25 PM.

    #2
    Creation Process

    More images of the creation process:

    Click image for larger version

Name:	braulio-fg-zbrushgrab.jpg
Views:	1
Size:	235.9 KB
ID:	1132107

    Click image for larger version

Name:	braulio-fg-hairexample.jpg
Views:	1
Size:	483.3 KB
ID:	1132108

    Click image for larger version

Name:	braulio-fg-lightsscreenshot.jpg
Views:	1
Size:	481.4 KB
ID:	1132109

    Click image for larger version

Name:	braulio-fg-innermouth.jpg
Views:	1
Size:	349.9 KB
ID:	1132110

    Click image for larger version

Name:	braulio-fg-ue4-screenshot-03.jpg
Views:	1
Size:	353.1 KB
ID:	1132111

    Click image for larger version

Name:	UE4_Screenshot_02.jpg
Views:	1
Size:	458.2 KB
ID:	1132676

    Click image for larger version

Name:	UE4_Screenshot_01.jpg
Views:	1
Size:	146.6 KB
ID:	1132677
    Last edited by BraV FG; 08-20-2017, 05:16 AM.

    Comment


      #3
      How you capure the video? Kinect?

      Comment


        #4
        cool
        capture face rig?

        Comment


          #5
          Yes I use a Kinect for Xbox 360 along with the software Faceshift, an old version I have from 2014. Sadly this software is no longer for sale, but another similar solution could be Brekel Face 2.0

          Comment


            #6
            Originally posted by sleepingdragon View Post
            cool
            capture face rig?
            Sorry, do you mean how I capture the movement? or how does the rig of the face (joints, blendshapes) looks like?

            Comment


              #7
              This looks great. For me I couldn't stop focusing on how still the nose is compared to the lovely natural 'twitching' of the rest of the face though. There are moments that would cause the nose to wrinkle a little bit, or the nostrils to flare and twitch. This isn't something that could be captured probably, but would need to be added after by the animator. Great job though

              Comment


                #8
                level God! that looks very awesome. please @bravFG, can you tell us how do you did the rig of face, how you organize the blendshapes and a peek view of joints.

                Comment


                  #9
                  Outstanding work! Congratz!

                  I loved the model, I would like to see another video with the main light moving around the head... because it looks like you are not using a mask to tell where subsurface will be more relevant (maybe you are using, but cant tell by the light position) as the ears looks too much solid look. As for the eyes, I really loved how the pupils contracted after a blink, thats hard to see around in many works... need a little bit of work on the moisture at them, they look a bit dry, besides the specular visible there... I really believe you will achieve what you want soon! Keep going and I hope see your work end results!
                  Nilson Lima
                  Technical Director @ Rigel Studios Ltda - twitter: @RigelStudios
                  Join us at Discord: https://discord.gg/uFFSEXY

                  UE4 Marketplace: Cloudscape Seasons
                  supporting: Community FREE Ocean plugin

                  Comment


                    #10
                    Originally posted by dannyflint View Post
                    This looks great. For me I couldn't stop focusing on how still the nose is compared to the lovely natural 'twitching' of the rest of the face though. There are moments that would cause the nose to wrinkle a little bit, or the nostrils to flare and twitch. This isn't something that could be captured probably, but would need to be added after by the animator. Great job though
                    Thanks, in the rig there's a subtle movements that are associated to other movements of the face. For example, when the mouth smiles the nose move a little bit. So when the animation is recorded, those secondary or associated movements also move along, I decide to leave them, but yeah I could also clean them up.

                    Comment


                      #11
                      Originally posted by michelart View Post
                      level God! that looks very awesome. please @bravFG, can you tell us how do you did the rig of face, how you organize the blendshapes and a peek view of joints.
                      Thanks so much! sure no problem, hope this image helps! Click image for larger version

Name:	RigScreenshot.jpg
Views:	1
Size:	384.1 KB
ID:	1132633

                      Comment


                        #12
                        Originally posted by NilsonLima View Post
                        Outstanding work! Congratz!

                        I loved the model, I would like to see another video with the main light moving around the head... because it looks like you are not using a mask to tell where subsurface will be more relevant (maybe you are using, but cant tell by the light position) as the ears looks too much solid look. As for the eyes, I really loved how the pupils contracted after a blink, thats hard to see around in many works... need a little bit of work on the moisture at them, they look a bit dry, besides the specular visible there... I really believe you will achieve what you want soon! Keep going and I hope see your work end results!
                        Thanks! I was using a translucency map to tell the subsurface where to be more relevant, but I also have several mask that controls parts of the face independently. One of those mask controls the ears, I could easily tone the roughness or transculency back up. Thanks for your suggestion on the eyes, I´ll work on that, eyes are a difficult subject. I animate the pupils via sequencer
                        Last edited by BraV FG; 08-18-2017, 05:38 PM.

                        Comment


                          #13
                          Do you plan to release the project file?

                          Comment


                            #14
                            Originally posted by DowntownSanDiego View Post
                            Do you plan to release the project file?
                            Not really, this its just a test for me, so no plan on releasing or selling or anything like that. But I was given the oportunity to create a breakdown involving each aspect in the creation of this test, you can read it here https://80.lv/articles/realistic-human-character-study/

                            Comment


                              #15
                              Here are some gifs to better understand how it all works:

                              Mocap recording:



                              Roughness control:



                              Normals control:



                              Mask control:

                              Attached Files
                              Last edited by BraV FG; 08-20-2017, 06:48 AM.

                              Comment

                              Working...
                              X