I checked in the curve editor and it looks like both keys are kept. This is beyond unintuitive. Is there a way to disable this?
