The problem was that Controllers only exist on the client that owns it and the server, so when I tried to do something on a controller that did not exist on a client nothing happened.
I fixed it by moving most of the nodes to the Character which does exist on all clients.