Yes, I had noticed 4.12 behaved different in PIE mode. 4.11 worked fine, so something was lost in translation.
It’s not a problem when you packaged the game, so it’s only an annoyance when you work in editor, but until I find out what changed in 4.12, you can use a workaround.
Open up advanced settings in play menu
Note the resolution of your new window size. (Or set your own)
Set the settings menu resolution to be the same in window mode. Press OK to save, and close PIE.
Now you should be able to playtest your game in PIE without it toggling/flickering between different window sizes.
You can modify the other settings.
If you want to test how your game behaves in different resolutions, you can use standalone mode for that.
In standalone game you can also properly go into fullscreen mode.