Adding onto this, unchecking
Editor Preferences -> Play -> Multiplayer Options -> Run Under One Process
will also fix this.
It seems like the onscreen print messages are on a per process basis. Although notably this will run much slower, as each window is running a different instance of Unreal. Much faster than packaging every time you want to test though!