So I’m running into the same issue as you when running it in an editor window and not a separate game process.
I’ve entered this as UE-24920 because larger resolutions should be working for an editor window render. However, for the time being, please use the “Use Separate Process” option and make sure you’re using -ForceRes in the Additional Command Line Arguments.
-.