Thank you for your reply! 
I’ve already switched from DX11 to DX12 (SM6) the same day I upgraded my project from UE5.0 to UE5.1 - and this crash started to happen after I make the switch to UE5.1
I’ve visited the thread you mentioned before posting here, and it seems the problem is either :
- Not switch to DX12 or
- MSI afterburner is messing with Unreal Editor
both are not applicable to my case, unfortunately 
A similar case I found was on UnrealSlacker Discord channel where the person posted the exact same crash report as mine - he was using DX12 with the latest Nvidia driver, however, no solution was given except suggesting he downgrade from DX12 to DX11 to prevent the crash from happening again.