So this is the last bit I’m gonna do on this. I wanted to get it to work on only one mesh so you didn’t need two copies of it (and the whole scene). It works in the video (minus the bugs), but it only scales the vertices if their scaled positions are inside the scope. I don’t think this will work on flat surfaces like planes because none of the vertices will ever be inside the scope. So I think the two mesh method works better.
What’s happening inside the scope is fixable (it’s just not masking/clamping any vertices that go outside the scope). But besides that, the video shows a big artifact that I think makes this method unviable: you can see the zoomed meshes/vertices in the shadows (this happens in the two mesh method, too). That’s because the mask & zoom are being applied to the material itself, not the screen. So anything that should only happen on the screen also happens in the world.
Here, inside the scope looks fine, but outside, it’s clearly not right.
Maybe somebody can improve this, but that’s it for me. Fun experiment, though.