Ah so I guess the trick is knowing that this falling state is happening because you have run up your loop, rather than something else (like manually jumping or running off the edge of something).
You might try tracing forward in the proposed velocity first, to see if you actually hit geometry that you care to use this with. So sweep a capsule forward a short distance and either check the type of the actor, physical material, impact normal, etc and then decide if you want to add this velocity or not.