Tried disconnecting both individually and testing. No difference in performance. The only thing that happened is it messed up how to portal looks. Disabled the teleport also but not change again.
I might have to come up with an alternate solution to doing this. This method is really expensive especially when you have other stuff going on in the scene.