Show Residuals

I have to revive this one because I still don’t understanding this weird behavior and it’s getting frustrating.

When I switch residuals on in each of the 4 views, it turns off seemingly by itself at random times.
View 1 and 2 seem to remember the setting better than 3 and 4.
It doesn’t seem to have to do anything with loading a new project.
Do I have to save a custom setting for this to be permanent?

Tips are greatly appreciated.