There is a tutorial on creating dynamic constraint on the wiki:
You will find in it informations about the classes you need : UPhysicsConstraintComponent - FConstraintInstance
Basic informations on constraint there: Physics Constraint Component User Guide | Unreal Engine Documentation
But, to be honest, i am battling with it a little, at least for initialization of physically based aactor.
Defered Attachment is the key, in this particular case.
For physics attached component, i used:
component->AttachParent = parent ; // Deferred attachment
instead of
component->AttachTo(parent); // direct attachment
Cheers