Warm up frames with Path Tracing?

Having what appears to be a warm up frame issue that only affects PathTraced renders. Specifically with the complex road material in the City Sample project.

No matter what permutation of warm up settings I try it ALWAYS takes roughly 20 frames for the cracks and blemishes in the road to appear.

If I keep identical warm up settings but switch to a Lumen/Deferred render, everything is fine. There must be some CVAR or warm up setting specific to Path Tracing I am unaware of.