How to modify jointpositions on simulated skeletal mesh while still receiving event hits

Hi, I am working on a professional project in which I want to move a real actuator with my hand to move a skeletal mesh in UE4. And if this virtual ‘physical’ skeletal mesh collides with something in the virtual environment, I would like to receive the force from the collision to apply force-feedback on my real actuator.
Currently I am applying the positions on the skeletal mesh using the animation blueprint but here I can only find a way to do this when the skeletal mesh is fully kinematic. I also tried the poseable mesh component to allow bone transformations to be driven by normal blueprint but I cannot find how to apply physics/collisions with this.
When the skeletal mesh is fully simulated, I cannot modify the position of the joint to the skeletal mesh.
I also tried flipping between simulated and kinematic but when the skeletal mesh goes back to kinematic, it goes back to its initial position and does not apply the movements that came from the simulated period one frame before.

I also already tried the “set all bodies below physics blend weight”, but with this method we cannot have a fully kinematic model and the collisions do not happen as consistently.

Any help would be much appreciated!!!