Prevent Interaction through actors

You should use different collision channels for your actors. For example you can use BlockAll collision for your walls to prevent any LineTrace pass through it.