Prevent an object from receiving captured reflections

Generally, ambient occlusion should cope with it to some degree. In addition, you could try using a postprocess volume with ambient cubemap override over your moving object. In addition, if you are using baked lighting, you need to be aware about indirect light samples distribution, as those are also source of light for your movable objects.