Interestingly, disabling Translucency in the viewport mostly fixes the issue (still not perfect). But thankfully this allows me to get work done for now
