Animated NVIDIA Flex soft body rig

Hey there,me again :slight_smile: The physic spiders are actually rly cool,had a simular idea when seeing my muk beckon without an animation,but never got to actually make use of it.About the blob thing ,sadly ur molt apporach doesnt rly work,BUT u can switch between status softbody and liquid with high viscosity in realtime,also particle shape emitters can be attached to work with flex,to get a work around.(hope i did express that understandable since english isnt my native language) You can see an example of that when looking closly at my snowman example which turns static mesh to flex liquid particles ,same can be done with a softbody,also just splitting the mesh into different flex components works pretty good:) btw if you want u can help me with something,im trying to recreate the official nvidia flex demo examples and some work arounds in unreal and im rly struggeling with the balloon (softboy surface) filled with liquids ,since A:i found no way to make cloth dismemberment (which is actually used in the demo on the surface of the balloon) working and B: Liquidparticles behave strange in a volume .also whats rly in my intress,did u find any good looking values for surface liquids yet? (dont care so much about fps lose) concering radius,smoothing and so on ,cause i also aked this a couple of times in the gameworks thread but never got an answer to it,however there are some examples which actually look pretty good :confused: (recreation of the bloody elevator scene for example)

Cool, I didn’t know about the particle shape emitters. Sorry, I haven’t tried softbody surfaces yet. I didn’t have much luck with liquid rendering, it always looks bubbly.

Hey,

I searched this subjects years ago.

The best ressources i could find :

Naturalmotion’s system, used in gta, star wars the force unleashed, etc.

And this is recent, i dont understand why they dont make this as a game middleware, seems like researchers do not really know what to do with their work lol, once the thesis is published, it is forgotten :

Flexible Muscle-Based Locomotion for Bipedal Creatures

This is 2 years old, but evolved neural networks for motion is a really old idea.

Most people are trying hard to take the problem of biped locomotion from the opposite side, using inverse kinematics to place foots or hands while not solving the real problem at all.

Just see the presentation of paragon a couple of weeks ago, how they managed foot placement, and root motion.

Balancing is the main challenge. It seems neural networks are doing a great job at it, in reality, this is how nature does it.

Some researcher are working hard and bringing a lot of innovations, but the industry isnt giving a try.

Naturalmotion’s samples are incredible, it’s probably what got me interested in articulated characters. Thanks for the links, I’ve never seen Tyler Streeter’s page before. It’s cool that he includes source code. I think I first saw the “Flexible Muscle-Based Locomotion for Bipedal Creatures” demo at one of my favorite sites (MvdP Projects & Publications) a while back. I’ve got third party libraries for CMA-ES and neural nets working in UE4 but haven’t had the chance to try using them to learn tasks.

Nvidia FleX animated rigid body

I was testing a few ideas and it looks like you can animate a rigid body to create effects similar to super strong motors. Animating springs seems to work well when creating more traditional style motors. I’ve only been able to get it working with the core API but there should be a way to do the same thing in the ext API (probably by editing the flexAssetExt).

In this video I just created a rigid body using the bone locations as particles. Then on each frame the rest positions are updated to match the animation. One important note is that you need to get the transforms from flexGetRigidTransforms() to pass into flexSetRigids().

The main difference between this method and attaching to bones is that the character can interact with the simulation e.g could swim though water or have a monster truck crushing cars. It should also interact with cloth correctly.

I might try the following:

  • Create a low res mesh with verts at the particle locations. Then animate the flex body to match the vert positions. It could also be more bouncy by reducing the rigid body stiffness setting. This would use less particles (only need outer surface) and interact well with water and cloth.
  • Use a skinny animated skeleton and attach a soft body to it. Objects can be stitched together by having the particles overlap and by creating a spring with zero distance.

A bit off topic, but I only found out about the inbuilt windows 10 video capturing today (Windows Key + G). It’s much better than what I was using before (Microsoft Expression Encoder 4).

The mesh version seems to work. In these videos I just changed the rigid body stiffness setting. I’ll probably try floating a character in water and multiple layers of clothing next. It should also be possible to fix the characters location and orientation in place by adding three fixed position particles.

This is pretty cool.

Is there anything we (the GameWorks team) can do in the FleX integration that would help you?

for peopel who use the unreal engine and prefer blueprints, well source code is very confusing for the most part and a tutorial would be very much apprechiated in setting up flex itself

Thanks for the reply. There are currently three main things that I can do in the core API that I haven’t been able to figure out in the extended API. I should probably download the source code for the extended API at some point.

  • It would be cool if I could use springs directly. I tried reusing some springs from a fake cloth asset (changed particle indices) but it didn’t seem to work. Springs could be useful for stitching objects together, creating joints / hinges, distance constraints and motors.
  • For the animated character I updated the rest positions so there’s a particle per vertex. This is impractical for a high res model, what I’d really like to do is use a soft body and animate its rest positions. I’m guessing that soft bodies work similar to rigid bodies with rest positions / shapes that modify the mesh using bones rather than verts. It would be great if I could edit these settings (might be able to already but I haven’t found them).
  • When creating assets it would be useful if I could send in some fixed particle positions e.g flexExtCreateRigidFromMesh. This could be used to stitch objects together. For example, a rigid body car and a soft body antenna could have three particles (a triangle on each object) linked with zero length springs.

I’ve been thinking about trying something like Samcon or Trajectory Optimization. It would be great if there was an easy way to save and continue from a current state (even if the simulation isn’t deterministic).

It looks like GameWorks supports a large set of features. It would be really cool (but unlikely) if it included something like AIWorks. CMA-ES and Neural Nets seem to be very popular in some of the articulated character animation papers I’ve been looking at. It would be useful if I had a library of common AI tools that ran on the graphics card (I should probably try to learn cuda one day).

Hi,
@DivideByZero

I have tested your method with great success.

But i have a question concerning softbodies and attach to rigid.

In editor, simulate mode, when i drag a rigidbody with a softbody attached to it, if for example i move too fast, or if i push it against something, the attached softbody starts to slide, and detach to the rigid body.
So i wanted to know if there is a way to lock the attachment totally. Because for an animated character, having the body slide away from the skeleton makes the entire system not usable.

I guess, since this is pbd, the particles are locked together due to overlapping, and when you tick the attach to rigid, it doesnt apply the constraint that would make them separate.
But is there a way in flex to lock them. i wonder.

And i have another question concerning the collision distance with softbodies.
The generation of particles volume/surface is automated. But i noticed, the particles are on the edge of the mesh so, if you take the particle radius, lets say 2.0.
If you set the collision distance to the minimum in the container : <= 0.1.

Then your softbody collision distance will have an offset of 1.1.
Because the outer particles are sitting on the mesh surface. Even if you set surface sampling to 0.0, and keep only volume sampling. Particles are still popping out of the mesh.

The only solution i can think of, is to scale these objects up. I tried a negative particle collision distance, but it wasnt a good idea lol.

I mean if you want to have a soft body animated hand, grabbing stuff around, your camera is close, and you see a big offset between fluids/and meshes. This is weird.
Flex looks good at distance. I hope, there are workarounds for these things i mentionned.

Also setting a smaller particle radius isnt an option, for softbody, it kills the frame rate. And causes tunnelling if you do not adjust the particle distance (but if you adjust you end up with 99999999999 particles on one object)
On fluids, it becomes too tiny.

Edit :
I took a look at the flex attach to rigid body implementation.
So it is just a bare copy of transforms from the rigid body to the particles.

in FlexComponent.cpp, line 894 :



const FVector AttachedPos = AttachTransform.TransformPosition(Attachment.LocalPos);

ContainerInstance->Particles[ParticleIndex] = FVector4(AttachedPos, 0.0f);
ContainerInstance->Velocities[ParticleIndex] = FVector(0.0f);


So i guess, since the particles position are forced to position, then it must be the softbody itself that becomes screwed up. I noticed that in simulate mode, when things go wrong, if i shake the rigidbody hard enough, the attached softbody returns to position.
The weird sliding effect may be related to the soft particles themselve, due to high forces, some of the links or constraint break, i dont know.

Hi, I had similar issues when I moved a character too fast or through a wall. I found that using larger rigid bodies like the “UE4 FleX rig 2” sample helped. Unfortunately, this would only be useful for rigging and it would lose most of its soft body properties. It also didn’t interact well with cloth and liquid (tends to pass through).

The particle size was also an issue because the size has to be fixed for all interactions (character, clothing, water). It would be cool if it supported variable particle sizes but I guess it wouldn’t work as well on the GPU. I was more interested in testing a full character so I used large particles. As you mentioned, this wouldn’t work well for small details like hands or the face.

Because of this I thought I’d try another approach in code. In the “FleX animated rigid bodies” samples I created the equivalent to an active ragdoll. It only needs particles on the mesh verts and is hollow so I can use smaller particles and keep detailed areas like hands. I was also thinking of using a smaller mesh for the particles (overlap). Cloth and liquid should also work (haven’t tried it yet). It should be possible to stitch objects together with springs e.g belly fat soft body, could bounce off ground from a fall. The main downside is that it doesn’t use the integrated branch so I lose most of the features. Its also very difficult to control (can pin areas but it could cause similar movement issues like attach to rigid). It might be interesting for basic characters like snakes.

Overall I think the first method mentioned in the tutorial would probably only be useful something like cutscenes with slow movements. It would be really cool if the integrated branch included FleX characters one day. It’s a shame that it has these amazing water, softbody and cloth features but you can’t have a character with two way interaction. I doubt it will be added however, as FleX is really only supposed to be used for secondary effects. I’m probably going to be working on some VR projects so I won’t be using FleX for a while.

In the datails pane of the material, “Usage” part, you need to select “Use with flex meshes”.

You probably know that by now, but maybe someone has that problem again.

I planned to create a destructible steel wall in unreal using flex. The goal was to be able to shoot the wall with a cannon and then see the wall deform, with several shots at the same point it should open a hole. I thought it would be easy to create the wall and convert it into cloth with tearing. The problem is that no mather what I do, the cloth never gets stiff like metal. And cloth is the only one with tearing. Anyone have any idea what I could do?

Looks like stiffness parameter are not working on cloth.

Creating a destructible steel wall with cloth sounds interesting. I’ve seen examples using plastic deformations on rigid bodies e.g metal barrel hits the grounds and it dents (maybe not supported in UE4?). I haven’t seen an example that would tear like cloth after a few hits. I think the stiffness setting might be the spring strength or stretchiness of the cloth. It might work if you could add shape constraints to the cloth (would probably require code).

I saw this awhile ago and thought it was really cool. I hope it’s included in a future version of Flex.

I was also testing motors for soft bodies in Unity recently. I’ll probably try it in UE4 at some point if I get the bone rotations working.
https://youtube.com/watch?v=aak0qt_pgMwhttps://youtube.com/watch/RHeS_TIR77M

good work!

That looks great. The first video is what I wanted, just without the elasticity. You have some tutorial about creating that motors?

This video at 0:43 does what I was trying:

Just read this in FLEX documentation: “The Flex UE4 integration does not currently expose all of the features in the Flex SDK, such as smoke simulation, or plastic deformation of rigid bodies. These features will be added in a future release.”
Looks like we need to wait.

why is lie?? i dont see flex in ue4 ???