Network Calling Function from Client on Server Problem

But the whole Reason why I want a PlayerController on my Pawn is because otherwise I cannot call the server RPCs. I basically want my setup to be 1 server and 3 clients where the game runs on the server and all actions from the clients are simply forwarded through RPCs to the Server.