Build a single-channel (grayscale, or alpha) mutable or render-text texture that expresses how much the wind is hitting various parts of your world.
Use the X/Y world coordinates, divided/offset to map UV 0-1 across your level, as an index into this texture.
Update the texture procedurally to be whiter where the wind is hitting.
Modulate your delta by the read result of this texture.
If you only ever have one (or maybe two) places that are “affected” by wind, you can instead have a texture parameter for “center of wind,” and perhaps another parameter for “radius of wind,” and you’d calculate how close the world point is to the wind point, and scale by that value. This doesn’t scale to multiple places of impact, though, as you’d have to do the distance calculation for each of the possible points, which gets costly pretty quickly.