I think this is a bug for UE4. I know this is related to screen DPI and Windows scaling. On my ROG G14 which has 14-inch 2560x1440 display, its screen DPI is high that Windows recommended 200% scaling. By using this recommended scaling value, I have the same behavior as OP. By changing this scaling to 175% will “solve” this problem but it is only a workaround. Because 175% will make other application like web browser or file explorer’s font size too small.
I found another method which is just uncheck “enable high DPI support” in editor preferences. However, this will make UE4 a lot blurry. So it seems there’s a bug in UE4’s high DPI support.