Hide the Actor only on the owning client.

or if you dont want it replicated (client only) use Visibility instead of SetActorHidden which is replicated by default i think