Figured out what my issue was. You have to make the camera movement inside a different sequence and then add it onto the “shots track” of another sequence. This video explains it: