@Everynone, Thanks for the detailed answer.
I tried the setup you mentioned, but the problem is that the objects are going through each other, like seen in your second video.
I do not want them to overlap, I just want the grabbed object to not exert any force or affect other objects in any way.
For example, see this video. I want the grabbed cube to behave with other cubes like it does with the environment. The other objects should simulate physics, but not receive force from grabbed object. Like when I grabbed the bottom one from the 2 stacked cubes, the top one should fall down as usual because it is simulating physics, I just don’t want it to react to being hit by the grabbed one.