Found another potential culprit which, once disabled, seems to make everything much more stable.
It is Windows Timeout Detection & Recovery (TDR). Basically, when Windows believes your Graphics Card has stopped working (hang/frozen), it will disable (aka remove) it and then re-enable it in rapid succession. This makes the GPU unavailable for a short time to any application, which consequently may crash.
There is a registry key which disables TDR. More information can be found here: