I actually did that as a proof of concept in UE4. I changed the LastFrameWeight based on camera movement. It worked suprisingly well!