Door open when I am not looking at the door !

Hello I am trying to open a door using press any button for example ( E ) and it will open door , I already make it successfully with trigger box, if player reach to door inside trigger box and press E button it will open, But the problem I am having is that when I am not looking at the door I can still open and close this door ,What is going on here and how can I fix it.
If there any tutorial regarding my issue it will be help .
Thanks in advance.

So the problem is that you only have to be intersecting a trigger in order for it to be activated, if you look into line tracing this will mean you have to be aiming at the trigger to be able to activate it. You can also set how long the trace should be so you would have to be right up close to the door handle to open it.

Line Tracing