I thought 100Mb is quite good – we aim for 200Mb for dedicated server memory.
Main focus is to reduce number of assets that have hard references.
For example, your main character blueprint may hold array of possible weapons. Each weapon could be a blueprint with a list of animations, sounds, particles and meshes that are needed while character has that weapon equipped. Dedicated server cooking removes some of the elements (eg sounds and particles) from build, so these wouldn’t use memory, but it is possible that your server doesn’t need meshes nor animations either (because they don’t have root motion) and are needed on the client only.
So, if character blueprint has TArray<AWeapon*> all such weapons with all assets that they refer to will be loaded into memory. If, however, you have TArray<TAssetPtr<AWeapon>> you can then decide when to load it into memory.
If this AWeapon class has code required to execute on dedicated server, then you can have such pointers to assets that are optional, and load them on the client only…
Further reading:
TAssetPtr_and_Asynchronous_Asset_Loading
Good luck!