Hello there @Sweenus987!
Checking around the community, and per UE’s documentation, the answer would be, that only the shader permutations that are actually used in the scene are compiled. The engine won’t compile every possible permutation by default. In your example with 24 possible permutations, if only 3 are used in the project, only those 3 will be compiled.
The engine tries to limit the number of permutations to only those that are likely to be used. A combination of static switches, quality settings, and platform features help determine which permutations are needed for a particular shader. This ensures that only permutations that will actually be used at runtime are compiled.
Further details can be found in the guide below: