So, after looking through how ScalabilityQuality levels are saved / loaded, I have a much clearer idea of how it actually works under the hood. Essentially these levels are loaded from different files in editor than in packaged builds.
In editor there is a temp save file somewhere along the lines of “%AppData%/UnrealEngine/VersionNumber/Config/EditorSettings.ini”. Which is then used for these settings in this menu here:
But when in a packaged build, default values are read from Project/Config/DefaultGameUserSettings.ini and can be set like this:
; These values are not used when in editor, only in packaged builds as the default values
; Our default values are currently set to High(2) for every quality setting
[ScalabilityGroups]
sg.ResolutionQuality=2
sg.ViewDistanceQuality=2
sg.AntiAliasingQuality=2
sg.ShadowQuality=2
sg.GlobalIlluminationQuality=2
sg.ReflectionQuality=2
sg.PostProcessQuality=2
sg.TextureQuality=2
sg.EffectsQuality=2
sg.FoliageQuality=2
sg.ShadingQuality=2
sg.LandscapeQuality=2
TLDR; You won’t see the default ScalabilityQuality levels loaded when testing in editor, but will see them used when creating a packaged build. Only the defaults, not any user-set values.