Announcement

Collapse
No announcement yet.

Any NPC - Character Customization Tool for UE4 - Featuring shareable Lipsync

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Any NPC - Character Customization Tool for UE4 - Featuring shareable Lipsync

    Hi guys,

    Here is a sneak peek of the tool I'm currently working on.

    Any NPC is a tool developed using Blueprints which feature the ability to create and customize a humanoid character.

    Is completely Blend Shapes based and currently it features a couple of different body types ( more to come ).

    Also one of the most usefull features ( especially for RPGs ) is the ability to share lipsync animation between different characters, which saves tons of time.

    The lipsync is driven by 38 blend shapes based on the IPA phonemes setup.
    The lipsync itself is not baked on the character skeleton but its driven externally.

    What you see is the first build/test of this tool, also the lipsync itself is not 100% correct ( 10 minutes cleanup is more then enough ).

    PS: The base mesh is the one from the plugin Species from Exocortex, its used just for testing purposes; the lipsync animation is done externally via a separate rig, its NOT created inside UE4 ( just to be clear ).

    If you have any questions feel free to post it here

    More features will come soon ( female version, zombies, military, setup for Faceshift and Faceware facial animation and so on )

    Last edited by Enter Reality; 12-31-2014, 10:41 AM.
    ENTER REALITY

    VR Solutions

    Contact us for more informations

    #2
    Very nice Morphing System. What's the plan for applying clothing/armor?
    GOD of DREAMs: A Unique FTPS/RPG Sandbox MMO

    Comment


      #3
      Hi TechLord,
      I'm thinking about that...it would be a matter of adding new morph to the existing ones and setting the visibility as well...
      I'm also working on a custom GUI for the customization itself and a random crowd generation with paths, but I still need to take a look at that stuff, since I never had experience with crowds, but I'll give it a try.

      Probably in the near future there will be:

      - Female version
      - Zombies ( well, its the HOT stuff right now )
      - Military
      - Medieval ( so armour )
      - Casual
      - Business
      ENTER REALITY

      VR Solutions

      Contact us for more informations

      Comment


        #4
        Hi Nicolas,

        Recently viewed a video for Exocortex Species. The blending methodology appears to be similar to Daz3D Genesis. Using Target Morphs to customize a character's base shape is a awesome form of customization, and I would like to see the technique applied to non-bipedal creatures, weapons, vehicles, furniture, trees, structures, etc.

        Daz3D offers Wearables (Clothes and Accessories) to auto-fit their Genesis models no matter proportion, so I speculate the clothing is also using blendshapes/morph-targets. I've seen interchangeable clothing applied in two ways:1) overlay to the base body mesh, 2) interchangeable mesh replacement on a Modular Pawn. Perhaps Modular Parts with Morph-Target Deformation offers the most flexible solution?

        The Modular Pawn is a popular method of creating customizable Characters, and I believe using morph-target will be equally as popular (sameeek4 is also developing a Character Customization System using Morph-Targets). I'm developing a method of procedural generation that can be used to customize entities. It would be nice to combine all of these techniques into a single tool.

        Adding a GUI is a good call. Can you elaborate on what you mean by custom? No UMG? I'm personally taking an agnostic approach, providing only a interface to communicate to a UI for in-game customization. Overall, I think a tool such this needs an Asset Supply to be successful and I believe this is why DAZ3D Genesis is so popular with their massive catalogue of Wearables. I don't know if you have intention of selling on the Marketplace, but, I see huge potential with providing Clothes and Accessories for Any NPC.
        GOD of DREAMs: A Unique FTPS/RPG Sandbox MMO

        Comment


          #5
          I've actually downloaded the Daz3D package and I will take a look at it in the near future, but yes, the tech is the same, is all blend shape based.

          I'm testing a kind of autofit tool using Softimage ICE ( very similar to the Blueprint system in UE4 ) which allows for geometry manipulation easily without any coding, and for me is a huge advantage.
          This tool will allow to easily transfer any kind of geometry ( for now human only ) from one to another so that I can create Blend Shapes in seconds rather then spending time modeling and adjusting everything.

          For clothing I was looking at Marvelous designer and also Daz3D...since I would like to include also Apex clothing support I need to study a bit better what's going on with it and see what could be the best solution for clothes.

          I really would like to explore the differences between the modular pawn setup and a Blend Shapes setup, since I guess one would be more memory heavy then the other...in case I will re-write the Blueprint setup or find an alternative solution.

          Daz3D would be a very good candidate because of the huge amount of stuff available, but if I remember correctly there is a fee to pay for game development ( 500$ ) and right now I cannot invest this money.
          Sorry, my bad, the GUI will be done via UMG, I need to work a bit on the graphics and figure out how to setup a random NPC generator which will include beard, clothes, glasses, hats, hairstyle without ******** the npc itself, menas without having conflicting results.

          I'm actually thinking to sell Any NPC as a tool on the marketplace once I figured out all the stuff mentioned above, but I already saw that the models available on the martekplace are quite expensive...and since this toll would solve quite lots of problems in terms of content creation I guess that I will propose this tool at 50$ more or less...also consider that I will include LODs as well, which done in Softimage takes 5 seconds ( god bless Softimage and thanks Autodesk for murder this fantastic software )

          Also the Lipsync tool will be sold separately, but it can be easily intergrated into Any NPC, since the base Blend Shape is the same...main point for the Lipsync tool is to provide very cheap lipsync animation ( 1$ for each word ) which would be perfect for RPGs or open world games.
          The tool is completely agnostic to the rig itself and it rely on Blend Shapes as well, and I'm expanding it so that it can be adapted to already available facial animation software, such as Faceshift, Faceware and so on.


          GOLEM looks great! really nice project! what about skin weights? I'm curious to see what can be done with different parts

          Sameek4 is doing a great job, he's a really talented guy and also his other tools ( road intersection tool ) looks fantastic!
          ENTER REALITY

          VR Solutions

          Contact us for more informations

          Comment


            #6
            I learn something new with each one of your post. Marvelous Designer looks amazing but not so sure about the Monthly Subscription $59.99/Annual Subscription $360.00. My Subscription Budget is maxed out and not a big fan of a Subscription for Tools. I'd rather pay the flat Lifetime $550.00. I see export options for obj and fbx. I foresee additional work involved to fit garments to base mesh and adding morph-targets, but, overall could be valuable in producing an infinite assortment of marketable Clothing Packs for Any NPC in the Marketplace. IMO, Any NPC would sell better in the Marketplace with a decent number of Clothing Extras available for purchase.

            I'm not certain if there's a standard for blendshapes/morph-targets when submitting to the Marketplace, but, there are considerations for the Skeletal Rig. Does the Any NPC BaseMesh use the Epic Biped Skeleton? By the way, I noticed in the Daz3D Morphing Vid all of the morphs are Biped, but, I could easily visualize Bipeds morphing into various Quadrupeds/Bird-like Creatures with additional blends extending vertebra and fore-limbs. Possibly an edge up for Any NPC on the competition

            I have yet to use Daz3D Genesis (I need to do this to at some point)...from what I've seen in their videos the base mesh is singular and Morph Sets and Clothing are modular. That's a pretty slick design and the Genesis SuperSuit is inspiration for other type of morphing clothing and accessories. Marvelous Designer + Daz3D SuperSuit = MightyMorphingPowerSuitDesigner? Add AutoFit + Apex Cloth support + In-game Editing/Customizing Interface and there's a Tool worth a subscription...hmmm

            In a Modular Pawn, separate meshes represent different shapes that are swapped out entirely. I fancy the idea of a hybrid system using modular morphing base mesh pieces. In theory, should allow greater flexibility and variation (ie: different morph sets per piece). The option to use a singular base should be available by default. Again, I'm looking for possible edge up for Any NPC on the competition.

            Lip Syncing and facial articulation is sorta of an expectation for characters in next gen games. I've seen it used in several games with narrative not just RPGs. I would vote for its integration into Any NPC more so as Developer expectation. Could very well be a highlight selling point.

            Thanks. A GOLEM Entity Body is the conglomeration of hundreds instanced static meshes. There is no skin.
            GOD of DREAMs: A Unique FTPS/RPG Sandbox MMO

            Comment


              #7
              Thanks for the comment

              Daz3D could very be the base for all the Any NPC blends, but still the license for game development is not perfectly clear...I guess I'll send an email directly to them explaining what I need to do and see the response...anyway the price is much better then the 3d scans of people, which are bloody expensive!

              Yes, the skeletal rig is based on the ART plugin from Epic, and I intend to add some more features to the base mesh, such as proxy bones for clothes and fat physic movement ( inspired by the amazing Judith Simantov The Last of Us tech demonstration and the brand new Uncharted 4 tech demo, so there will be a couple of variations for the skeletal rig itself in order to use the proper one for each character...also breast physics ( ala Witcher 2 ) will be implemented

              Still lot of stuff to check and still some RnD is necessary, but the base layer is now done, so it'll be just a matter of adding content and make it the best and more efficient way possible.

              Facial animation is something which I really would like to work in the future, and for now lipsync is just the beginning simply because the results are pretty good and, as said, facial animation is quite expensive, so I will also add a complete expression set for the face ( blend shape based, but I already developed an additional system which uses bones as well ) in order to have proper facial expressions with amazing results...
              Also I'm re-learning Zbrush ( had to switch from Softimage to Maya, its still painfull ) and currently the next demo will be way more polished and will show off also a very nice trick, which is Dynamic normal map based on expressions
              The video looks like ****, but it gives the idea...also the fast way to achieve that is via Blend Shapes, but I also found a method to use it with traditional joint based facial rig.

              Lots of stuff to do, lots of things to test and lots of stuff to learn...hopefully by combining all of those things something great will come out

              No skin on Golem? interesting approach, I'm really curious to see the progress on that
              ENTER REALITY

              VR Solutions

              Contact us for more informations

              Comment


                #8
                I applaud you sir...absolutely wonderful

                Streams: www.Twitch.tv/BlinkTheory
                Patreon: www.Patreon.com/BlinkTheory

                Comment


                  #9
                  Nicolas I viewed several of your Youtubes. You obviously possess significant experience with 3D Modeling & Animation. These are Skillsets I desire to improve on, being a code-centric gamedev. My lack of 3D Characters for my FP Hack & Slash, and skill to create them myself is actually what inspired the GOLEM ECS. This discussion has also been inspirational and informative. I'm currently using Shapes Components: Box, Capsule, Sphere in the GOLEM Fragment Attachment process. The addition of Static Mesh and Morph-enabled Mesh components to this process would provided even greater variation.

                  I viewed the videos on the Last Of Us Tech and Uncharted Tech, and their techniques look incredibly complex (and intimidating). I hope there are easier ways to pull this level of realism in UE4 for much smaller teams. I have a vague understanding of proxy bones and how they can be applied to clothing from working with the UE4 Owen Skeletal Asset. Perhaps UDK's workflow concept of a Master Skeleton is applicable?

                  My thoughts on Clothing for a Morph-enabled Entity is a 100% Apex-driven Cloth overlay that physically reacts to the Character's base mesh surface. In fact, the more I think about this subject, the more I see value in a in-game garment/wearables auto-fit & customization system. In a majority of games, humanbeings are often clothed, there is rarely a base mesh displayed. In some cases creatures are wearing armor etc. Such as system could be applied to any type of character, no matter the construction methodology used.
                  GOD of DREAMs: A Unique FTPS/RPG Sandbox MMO

                  Comment


                    #10
                    Thanks for the kind words TechLord

                    Well, not being able to code...well anything actually was a critical point to switch to UE4 in order to develop some of the ideas which without coding would be impossible for me to create.
                    To be honest the art and polish of the 3d content is something that, regarding gameplay, could be easily added later, simply because the highlight would be the Blueprint/coding itself which allows to do a specific thing, so if you want you could begin start learning Maya/Max to improve your skills, but for testing purpose I would storngly suggest to use free asset from the store or create something basic, you don't need always shiny stuff to wow people

                    A genius on this forum ( Rama ) create usually small gameplay videos with his new stuff, the art is "normal" for what he need to show and it does the job perfectly

                    Regarding the Master Skeleton this is the way I'm achieving small variations on the skin and proxy bones themself, means that I will add a couple of extra bones in some areas ( breast, belly, biceps ) using the ART plugin, which is pretty handy and not so impossible to use...it'll be just a matter of having proper skin weighting which can be adapted between the morph variation.
                    Also regadring the proxy bones, a good example of cloth behaviour can be seen inside the Matinee fighting sequence, if you export one of the main dudes you can see that the rig itself has proxy bones all over the cloth ( that kind of skirt, I cannot described it better ) and I think that the guys at Epic used the nCloth plugin ( or Syflex ) inside Maya and bake the simulation results on the cloth itself, so basically the bones are completely passive and they're driven by the simulation.

                    The only problem which I didn't test right now regaring the Apex cloth is that you should have a proxy mesh which will behave as a collision for the cloth itself on the character, and as far as I remember those are not morphable, or better yet I don't know how to access those meshes when the Skeletal rig is imported into UE4...is something that I need to clarify both in Maya and UE4 and I need to do some research on the forum itself, so eventually I'll find what I'm looking for.

                    Quick update on Any NPC:

                    - Added Scale for root bones in order to set character height ( from child to giant )
                    - Starting UMG GUI menu
                    - Improving thelipsync and phoneme shapes in order to achieve better results
                    - Adding expressions for Faceshift and Faceware setup ( this will take a bit longer )
                    - Cloth setup with dynamics

                    New video in a couple of days
                    ENTER REALITY

                    VR Solutions

                    Contact us for more informations

                    Comment


                      #11
                      Small update:
                      Character scale while applying blend shapes

                      Optionally ( need to check some stuff, but 99.9% sure ) I'll include the possibility to scale single bones also so that you can create a non human shape directly inside the Unreal Engine editor.

                      ENTER REALITY

                      VR Solutions

                      Contact us for more informations

                      Comment


                        #12
                        Originally posted by Nicolas3D View Post
                        Small update:
                        Character scale while applying blend shapes

                        Optionally ( need to check some stuff, but 99.9% sure ) I'll include the possibility to scale single bones also so that you can create a non human shape directly inside the Unreal Engine editor.
                        Hey Nicolas,

                        The morphing is very smooth. Its a little tough to gauge the scaling without a reference. Does scaling the single bones equate to --> morphing into various Quadrupeds/Bird-like Creatures with additional blends extending vertebra and fore-limbs?
                        GOD of DREAMs: A Unique FTPS/RPG Sandbox MMO

                        Comment


                          #13
                          Yep,

                          About the height scale I didn't have the time to setup the UMG, still learning how to make it work as I want

                          Anyway yes, since you have the ability to access bones transforms you'll be able to scale/rotate what you want, of course I need to expose those values for the user to modify them, but overall if you want to create a warewolf/quadruped or non human you'll be able to do that.
                          Alternatively, or as an additional option, I'll probably include a different skeletal rig ( probably more then one ) for particular cases ( tail for quadrupeds, ponytail for hair, bpnes for chains for armor, and so on ).
                          ENTER REALITY

                          VR Solutions

                          Contact us for more informations

                          Comment


                            #14
                            Another small update showing the blend shapes in action controlled by the user.

                            The main reason why I didn't create a UMG menu with sliders ( at least for now ) is that UMG works only when you're in Play mode, so it becomes annoying to remember every time to hit the play button so you can modify the character in the scene.
                            Using this method you're able to modify the character in both "non-play" mode and in play mode





                            PS: Female base character coming soon
                            ENTER REALITY

                            VR Solutions

                            Contact us for more informations

                            Comment


                              #15
                              WIP of the female version.

                              The Blueprint will be the same as the male version since its already setup, but it will include more features...

                              Next up, UMG customization slider/setup

                              Click image for larger version

Name:	Female_BS.JPG
Views:	1
Size:	70.6 KB
ID:	1065884
                              ENTER REALITY

                              VR Solutions

                              Contact us for more informations

                              Comment

                              Working...
                              X