No announcement yet.

Animated NVIDIA Flex soft body rig

  • Filter
  • Time
  • Show
Clear All
new posts

    That's F-ing awesome!

    Damnit, times like these I feel dumb.


      Been showcasing this since october 2015,basicly works on everything thats mesh or even bound particle based all started with a blobby muk testing lol,all i got to say is great job overall rly like the idea of muscle simulation atm im working on a real blob character actually using both the fluid particles + the softbody simulation since both are particle based anyway these work pretty good togehter,ill post some picture once it reaches a state ,im happy with and i hope for an update on ur side (rly liked the sneak peek into ur ideas)


        A combination of fluid and soft bodies sounds super cool. After seeing your example I thought I'd try making a mold and filling it with a high viscosity liquid. Then I could remove the mold and maybe it would fall like jelly. As you can see in the videos it didn't really work.

        I'm currently taking a break from FleX. The last thing I checked was how to attached the character mesh to the muscles. UFlexAssetSoft::ReImport() calls flexExt.h, flexExtCreateSoftMeshSkinning(). The particles look like they are attached to the mesh using bones and GPU skinning (FFlexGPUVertexFactory). I haven't tried it but I could wait for the muscle to move into place and take a snapshot of the bones to import into blender. In blender the character mesh could be skinned to the soft body bones and exported back into UE4. Then when the game runs I could just update the skeletal mesh bones to match the soft body bones.

        Completely off topic, but I'm currently interested in physics characters. Here's a test video I made using the infinity blade SK_Greater_Spider. The physics is just driven by the animation. I was surprised at how well the character can move using just the animations.

        Biped characters still need some kind of balance system. It looks like it can get enough torque to do jumps.


          @DivideByZero - That's awesome!

          How do you set it up to drive the physics bodies with the animation, anyway?

          Also, I've known about and wanted to set something like this up for years but it was never terribly feasible for me:

          Might give you some ideas how to do a good balancing system.
          Storyteller - An immersive VR audiobook player

          Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes


            I was really excited when I finally got the physics to match the animations. For anyone who hasn't tried this, some of the main issues are a lack of torque, the joints separating and it's really difficult to control. The main problems with control are that motors don't take into account internal forces (e.g moving the forearm will require a counter torque on the upper arm) and external forces (gravity, collisions). Normally you would create PD controllers and try tweaking the values to reduce effects like oscillations.

            I read a GDC paper called "Practical Physics for Articulated Characters" and it talks about using Featherstone for physics.
            I tried a few options that support Featherstone.
            • Simbody: Great for muscle simulation and it's easy to integrate into UE4. Has an odd method to handle collisions e.g center of triangles, so a cube needs to be high poly to avoid passing through it.
            • DART: Mainly for robotics. I wasn't able to get it working in UE4.
            • Bullet: I got it working in UE4 but I couldn't get stable results.

            I think you really need a good understanding of robotics and physics if you want to use Featherstone. It was all a bit beyond me, even the beginner stuff like jacobian matrices is a nightmare. I was about to give up when I found a really easy answer. PhysX already has its own solution for articulations. It is marked as experimental but the results are great. You can tell it what percentage of internal and external forces you want to counter then just set a target pose. And it also prevents the joints from separating. There aren't any existing components that I know of so you need to access the PhysX SDK directly. Because it can handle internal / external forces it makes controlling the character from an animation really easy. If you counter all internal and external forces you could pick a fixed pose and drop the character on the ground and it will look like one fixed rigid body (as seen in the biped clip, no bending knees). It's also really easy to match the animations, I just set the target pose to the same value as the animation pose.

            Thanks for the link to the balancing method. I actually tried this a couple of months ago but I couldn't seem to get it to work. I'm thinking I might try SIMBICON or some of the training examples I've seen.


              Hey there,me again The physic spiders are actually rly cool,had a simular idea when seeing my muk beckon without an animation,but never got to actually make use of it.About the blob thing ,sadly ur molt apporach doesnt rly work,BUT u can switch between status softbody and liquid with high viscosity in realtime,also particle shape emitters can be attached to work with flex,to get a work around.(hope i did express that understandable since english isnt my native language) You can see an example of that when looking closly at my snowman example which turns static mesh to flex liquid particles ,same can be done with a softbody,also just splitting the mesh into different flex components works pretty good btw if you want u can help me with something,im trying to recreate the official nvidia flex demo examples and some work arounds in unreal and im rly struggeling with the balloon (softboy surface) filled with liquids ,since A:i found no way to make cloth dismemberment (which is actually used in the demo on the surface of the balloon) working and B: Liquidparticles behave strange in a volume .also whats rly in my intress,did u find any good looking values for surface liquids yet? (dont care so much about fps lose) concering radius,smoothing and so on ,cause i also aked this a couple of times in the gameworks thread but never got an answer to it,however there are some examples which actually look pretty good :/ (recreation of the bloody elevator scene for example)
              Last edited by Daniels Channel; 02-18-2016, 09:24 AM.


                Cool, I didn't know about the particle shape emitters. Sorry, I haven't tried softbody surfaces yet. I didn't have much luck with liquid rendering, it always looks bubbly.



                  I searched this subjects years ago.

                  The best ressources i could find :

                  Naturalmotion's system, used in gta, star wars the force unleashed, etc.

                  And this is recent, i dont understand why they dont make this as a game middleware, seems like researchers do not really know what to do with their work lol, once the thesis is published, it is forgotten :

                  Flexible Muscle-Based Locomotion for Bipedal Creatures

                  This is 2 years old, but evolved neural networks for motion is a really old idea.

                  Most people are trying hard to take the problem of biped locomotion from the opposite side, using inverse kinematics to place foots or hands while not solving the real problem at all.

                  Just see the presentation of paragon a couple of weeks ago, how they managed foot placement, and root motion.

                  Balancing is the main challenge. It seems neural networks are doing a great job at it, in reality, this is how nature does it.

                  Some researcher are working hard and bringing a lot of innovations, but the industry isnt giving a try.
                  Last edited by GameKittens; 02-28-2016, 07:53 AM.


                    Naturalmotion's samples are incredible, it’s probably what got me interested in articulated characters. Thanks for the links, I’ve never seen Tyler Streeter’s page before. It’s cool that he includes source code. I think I first saw the “Flexible Muscle-Based Locomotion for Bipedal Creatures” demo at one of my favorite sites ( a while back. I’ve got third party libraries for CMA-ES and neural nets working in UE4 but haven’t had the chance to try using them to learn tasks.


                      Nvidia FleX animated rigid body

                      I was testing a few ideas and it looks like you can animate a rigid body to create effects similar to super strong motors. Animating springs seems to work well when creating more traditional style motors. I've only been able to get it working with the core API but there should be a way to do the same thing in the ext API (probably by editing the flexAssetExt).

                      In this video I just created a rigid body using the bone locations as particles. Then on each frame the rest positions are updated to match the animation. One important note is that you need to get the transforms from flexGetRigidTransforms() to pass into flexSetRigids().

                      The main difference between this method and attaching to bones is that the character can interact with the simulation e.g could swim though water or have a monster truck crushing cars. It should also interact with cloth correctly.

                      I might try the following:
                      • Create a low res mesh with verts at the particle locations. Then animate the flex body to match the vert positions. It could also be more bouncy by reducing the rigid body stiffness setting. This would use less particles (only need outer surface) and interact well with water and cloth.
                      • Use a skinny animated skeleton and attach a soft body to it. Objects can be stitched together by having the particles overlap and by creating a spring with zero distance.

                      A bit off topic, but I only found out about the inbuilt windows 10 video capturing today (Windows Key + G). It's much better than what I was using before (Microsoft Expression Encoder 4).



                        The mesh version seems to work. In these videos I just changed the rigid body stiffness setting. I’ll probably try floating a character in water and multiple layers of clothing next. It should also be possible to fix the characters location and orientation in place by adding three fixed position particles.


                          This is pretty cool.

                          Is there anything we (the GameWorks team) can do in the FleX integration that would help you?


                            for peopel who use the unreal engine and prefer blueprints, well source code is very confusing for the most part and a tutorial would be very much apprechiated in setting up flex itself


                              Thanks for the reply. There are currently three main things that I can do in the core API that I haven’t been able to figure out in the extended API. I should probably download the source code for the extended API at some point.
                              • It would be cool if I could use springs directly. I tried reusing some springs from a fake cloth asset (changed particle indices) but it didn’t seem to work. Springs could be useful for stitching objects together, creating joints / hinges, distance constraints and motors.
                              • For the animated character I updated the rest positions so there’s a particle per vertex. This is impractical for a high res model, what I’d really like to do is use a soft body and animate its rest positions. I’m guessing that soft bodies work similar to rigid bodies with rest positions / shapes that modify the mesh using bones rather than verts. It would be great if I could edit these settings (might be able to already but I haven’t found them).
                              • When creating assets it would be useful if I could send in some fixed particle positions e.g flexExtCreateRigidFromMesh. This could be used to stitch objects together. For example, a rigid body car and a soft body antenna could have three particles (a triangle on each object) linked with zero length springs.

                              I’ve been thinking about trying something like Samcon or Trajectory Optimization. It would be great if there was an easy way to save and continue from a current state (even if the simulation isn’t deterministic).

                              It looks like GameWorks supports a large set of features. It would be really cool (but unlikely) if it included something like AIWorks. CMA-ES and Neural Nets seem to be very popular in some of the articulated character animation papers I’ve been looking at. It would be useful if I had a library of common AI tools that ran on the graphics card (I should probably try to learn cuda one day).



                                I have tested your method with great success.

                                But i have a question concerning softbodies and attach to rigid.

                                In editor, simulate mode, when i drag a rigidbody with a softbody attached to it, if for example i move too fast, or if i push it against something, the attached softbody starts to slide, and detach to the rigid body.
                                So i wanted to know if there is a way to lock the attachment totally. Because for an animated character, having the body slide away from the skeleton makes the entire system not usable.

                                I guess, since this is pbd, the particles are locked together due to overlapping, and when you tick the attach to rigid, it doesnt apply the constraint that would make them separate.
                                But is there a way in flex to lock them. i wonder.

                                And i have another question concerning the collision distance with softbodies.
                                The generation of particles volume/surface is automated. But i noticed, the particles are on the edge of the mesh so, if you take the particle radius, lets say 2.0.
                                If you set the collision distance to the minimum in the container : <= 0.1.

                                Then your softbody collision distance will have an offset of 1.1.
                                Because the outer particles are sitting on the mesh surface. Even if you set surface sampling to 0.0, and keep only volume sampling. Particles are still popping out of the mesh.

                                The only solution i can think of, is to scale these objects up. I tried a negative particle collision distance, but it wasnt a good idea lol.

                                I mean if you want to have a soft body animated hand, grabbing stuff around, your camera is close, and you see a big offset between fluids/and meshes. This is weird.
                                Flex looks good at distance. I hope, there are workarounds for these things i mentionned.

                                Also setting a smaller particle radius isnt an option, for softbody, it kills the frame rate. And causes tunnelling if you do not adjust the particle distance (but if you adjust you end up with 99999999999 particles on one object)
                                On fluids, it becomes too tiny.

                                Edit :
                                I took a look at the flex attach to rigid body implementation.
                                So it is just a bare copy of transforms from the rigid body to the particles.

                                in FlexComponent.cpp, line 894 :
                                const FVector AttachedPos = AttachTransform.TransformPosition(Attachment.LocalPos);
                                ContainerInstance->Particles[ParticleIndex] = FVector4(AttachedPos, 0.0f);
                                ContainerInstance->Velocities[ParticleIndex] = FVector(0.0f);
                                So i guess, since the particles position are forced to position, then it must be the softbody itself that becomes screwed up. I noticed that in simulate mode, when things go wrong, if i shake the rigidbody hard enough, the attached softbody returns to position.
                                The weird sliding effect may be related to the soft particles themselve, due to high forces, some of the links or constraint break, i dont know.
                                Last edited by GameKittens; 09-22-2016, 11:13 AM.