Do you have to process interaction on the server?
I’d suggest managing all UI stuff locally, this way there will be no mix-up with controllers.
And only send the necessary interaction data to the server when needed.
I don’t know your setup, so it’s hard to be more specific.