In RayTracingHitShaders.usf, on the GetMaterialPayload() function, MaterialAO is only correctly populated when taking the adaptive Substrate path, but missing on the Substrate disabled or blendable GBuffer path.
I fixed this issue locally like this:
/**
* Set material attributes for full materials
**/
Payload.TranslatedWorldPos = DFFastToTranslatedWorld(MaterialParameters.AbsoluteWorldPosition, ResolvedView.PreViewTranslation);
Payload.WorldNormal = WorldNormal;
Payload.Radiance = EmissiveColor;
Payload.BaseColor = BaseColor;
Payload.Specular = Specular;
Payload.Roughness = Roughness;
Payload.Metallic = Metallic;
Payload.GBufferAO = GetMaterialAmbientOcclusion(PixelMaterialInputs);
Steps to Reproduce
With Substrate disabled or set to blendable GBuffer format (SUBSTRATE_GBUFFER_FORMAT == 0), add an AO output to a Material and notice that Payload.GBufferAO is not populated (zero).
Let’s assume, we set it as you suggested, the issue is that GBufferAO is never actually stored. FMaterialClosestHitPayload is packed into a FPackedMaterialClosestHitPayload struct, which does not have space for storing GBufferAO. So despite being set, it the value will be invalid if you plan to use it as hit payload after a ray query.
We’ll see how the quantization looks like. Might not be something you want to adopt, but we’re currently using GetGBufferDataFromPayload(), which assumes the value is populated.