Data Layers set to unloaded show even when unloaded

It appears the issue is that the initial runtime state setting was somehow changed in the published island.
I managed to fix the issue by just republishing the island with no modifications.