So you are doing it correctly with regards to the second picture. A single value (Scalar) is coming out of the vertexcolor node to LERP between two 3vecs (Color-information).
What I CANNOT tell you is what the mesh might already have insofar as colors stored in the red-vertex-channel, the green, etc, etc.
What I said before is the ideal, but you will have to control the actual colors. I have no idea if the red-channel or whatever is suitable for what you want to do, just HOW it could/would be done.
I’m quessing quixel has some predefined information on the vertex-colors, so if you wanted to co-opt them for your own purposes, you’d have to paint them yourself, either in an external program, or in Unreal.
To-wit:
Pick a mesh, and enter paint-mode:
So, for example, this quixel rock I exported a while-back is completely-full in the red-channel. NOTE: I might have done that myself, no idea how it originally exported over…
As far as the top picture, it’s too fuzzy so I cannot make out the function-names, but if you are using different colors for different alphas (as you nominally WOULD be doing) it’s likely on-point insofar as the concept. Again, if the mesh already has predefined vertex-coloring, it’s more your logic is sound, but garbage-in-garbage-out.
Check your colors and go from there.