Quoted from here:
When these subsystems are initialized, it is also possible that certain values may not be available as expected for server/client instances. For example, a common problem is that the instance’s NetMode may not be correct when certain subsystems are initialized. This can cause issues if a subsystem has code that should only be run on a server or on a client during initialization. Furthermore, this makes it very difficult to control whether a subsystem is only spawned on a client or server, as during initialization there is not a good way to tell which one the instance will be.
Thus, I would destroy the subsystem on server at a later point in time when there is a valid world and I can check IsNetMode(NM_DedicatedServer).