Troubles with pivot changing in Matinee

Hi! I am currently learning how to animate a door opening and am using the Matinee to do so. However, the pivot for my door always changes to another spot whenever I add key. Any help would be appreciated!