How to target specific colors in a Color Mask?

This is a bad way of doing things. A single channel in an RGB texture contains 256 possible values (expressed in a 0-1 range in unreal). All of your colors could easily fit in just one channel of your image alone (think dark red vs light red).
A single bitmask node can then test if the pixel value matches the specified value. This will free up your other two channels for other things (like a roughness map for example).

What you’re trying to do is possible but it’s a waste of shader instructions and time vs a grayscale mask.