Why doesn't AutoReimport work with a Data Source Folder

We recently set up the “Data Source Folder” in the editor preferences so that Uassets can use relative file paths so as to be more portable between team members. It seems that doing so broke automatic asset reimporting that worked prior.

Is there something else you have to do along with setting the data source folder in order to preserve the auto reimport functionality?