They actually do since DirectX 10 (iirc) but you still have to specify the feature level (DirectX version) to use. If some game has code to automatically pick the highest available feature level it could pick DX12 without having to specify it explicitly.
However I believe the important features like the closer hardware access will require specific code so it’s unlikely you’ll see much of an improvement even though it’s running on DX12 in these cases.