Chest inventory opens anywhere not just at chest.

Are you doing any check to make sure the player is in the vicinity of chest? Usually with the ‘press E’ system, that would be overlap.