Yes, this is a bug, unfortunately. I have already reported it to Epic some months ago:
So upvote the post or comment it so that Epic knows there is “community interest” in this getting fixed ![]()
For now, a simple workaround is to not use Alt+Enter for fullscreen but instead the console command r.setRes WIDTHxHEIGHTf, for example “r.SetRes 1920x1080f”.