Loading freeze due to player controller replication

We have a very long (15-20 seconds) freeze of the main thread during map loading. After some debugging we figured why it happens.

The level loading is asynchronous and doesn’t block the main thread but during this time we get player controller replication command from the server. And when the player controller is being created in the main thread, it stops the async loading thread and flushes all the currently queued streaming packets which causes this long freeze.

So the question is, is there a way to avoid this long freeze somehow?