Create separate replicated actor for each client?

As the title suggests, I’m looking to have separate replicated actor for each client, where the actor is shared only between that client and the server (similar to the player controller), and the server can use those to talk to each client individually.

I know I could use the player controller, but there are some architectural constraints that would make this a bit annoying.

I.e. on server: ClientActorMap.Find(SomeID)->SendMessage(SomeData);

Any advice is appreciated!