Something changed in more recent Unreal versions (5.5 ish), appears that turning off this option in project settings itself has no affect as the underlying config variable was renamed or something. It’s now in DefaultEditorPerProjectUserSettings.ini and you add the following lines.
[/Script/UnrealEd.LevelEditorMiscSettings]
bCheckReferencesOnDelete=False