Issues with WP HLODs and bCastHiddenShadow

Hi,

We have an issue with custom HLOD builder where we want to use a lower polygon mesh for the shadow caster only for performance, while rendering a moroe detailed mesh for the main view. We accomplish this by putting two mesh components into the HLOD: One visible component with bVisible=true and CastShadow=false, and one shadow component with bVisible=false and CastShadow=true and bCastHiddenShadow=true

The problem we encounter is that the implementations of IWorldPartitionHLODObject::SetVisibility just toggle the bVisible flag on the components. This causes the invisible shadow casting mesh to be shown, but it also means that removing the HLOD by hiding it doesn’t hide the shadow caster due to bCastHiddenShadow keeping it visible in shadow rendering. Using bHiddenInGame=true instead of just bVisible avoids having the shadow mesh being forced visible, but it doesn’t fix the issue with rendering double shadows. The shadow casting only component in the HLOD casts shadows even when the HLOD should be disabled.

We noticed there is an existing FPrimitiveSceneProxy::bForceHidden flag which seems to be used by the LevelStreaming code, specifically to avoid issues such as these. Could HLOD visibility also use this flag instead of bVisible or is there a reason it does not?

Also, in terms of instanced HLODs, it seems like FISMComponentDescriptorBase ignores some flags from the source component such as bAffectIndirectLightingWhileHidden which also exhibit this same issue which causes them to be lost when building HLODs or PackedLevelInstances. This also includes other settings like DetailMode. Was this an intentional choice or an oversight?

Thanks,

Lucas

Steps to Reproduce
Create WP HLOD using bCastHiddenShadow either via a custom WP builder or using AWorldPartitionCustomHLOD which uses a flag like bCastHiddenShadow or bAffectIndirectLightingWhileHidden

Observe that HLOD visibility does not change correctly when HLODs are shown or hidden.

This repro project uses a AWorldPartitionCustomHLOD since it was simpler create a similar HLOD without needing to include our custom HLOD builder, but the same issue occurs on a standard AWorldPartitionHLOD.

Hey Lucas,

Thanks for reporting this and for the detailed description. I can confirm that I can see the same behavior locally when using AWorldPartitionCustomHLOD actor with bCastHiddenShadow. I’ll look into this and get back to you with some answers.

Thanks,

Andrzej

Hey Lucas,

Apologies for the delay.

Regarding the HLOD visibility - bForceHidden flag is used to ensure that level visibility is “atomic”, so that all components are made visible/invisible at the same time. Because of that and potential hidden implications, I don’t think I’d recommend using this flag for HLOD purposes.

In your case, the best solution might be to provide SetVisibility override. You could then make sure in your implementation that all flags are set properly, for example you could toggle bCastHiddenShadow as appropriate and make sure that your shadow component is never made visible.

Regarding the FISMComponentDescriptorBase ignoring some flags - support for bAffectIndirectLightingWhileHidden was added in CL 27067765 (Aug 2023) and for DetailMode in CL 25848314 (Jun 2023), so that’s been there for a while. Just to be sure I did a simple test and they were preserved in my test case.

If that’s not the case for you and you found a scenario where the flags are not preserved, please let us know and provide some details, so that we can investigate.

Thanks,

Andrzej

Hey Lucas,

Sure, I was just trying to explain why bForceHidden was added to the engine and what it’s used for.

In any case, changing the default implementation of SetVisibility would be a significant change that could affect existing projects relying on the current behavior.

As I mentioned in my previous message, you can look into overriding SetVisibility with your own implementation to handle the flags according to your needs.

Regarding FISMComponentDescriptorBase, I’m glad we got that sorted out.

Thanks,

Andrzej

Yes, I was referring to IWorldPartitionHLODObject::SetVisibility.

And you’re right that the suggested workaround might not work with FastGeo. I’ll need to take a closer look to see if there’s anything we could do there.

I understand your point of view, why it might feel inconsistent and why you think not using bForceHidden might be a bug. At the same time, having HLOD components with bCastHiddenShadow is not really a common or standard use case for the HLOD system. That’s why we’re trying to avoid making fundamental changes to support this use case, and why I was suggesting alternative approaches to help you achieve your goals.

I’ll review this again with the team and get back to you once I have more information.

Hey Lucas,

I talked to the team about this and your best option at the moment is to modify FFastGeoHLOD::SetVisibility. You could use something like this

ForEachComponent<FFastGeoStaticMeshComponentBase>([](const FFastGeoStaticMeshComponentBase& SMC)and adjust component flags as needed.

Currently there’s no easy way to do it without making an engine change, unless you fully subclass the FastGeo CellTransformer, which seems a bit overkill.

Hope that helps.

Thanks,

Andrzej

I see, yeah, that might require some additional changes to FastGeo.

Another option (and I understand it’s not ideal), could be to exclude your custom components from FastGeo Cell Transformer.

Hi Andrzej,

I’m not totally following you here about the atomicity of the flag, both bForceHidden and bVisible are per-FPrimitiveSceneProxy the only difference is how they are handled by the render, bVisible mostly excludes the component from the main pass (and by default some other passes like shadow casting based on other flags), whereas bForceHidden excludes the component from all rendering operations completely. That seems like what HLODs are attempting to do for actors that should be loaded but unused, there isn’t a difference between what granularity bVisible or bForceHidden apply to, there is just not an existing API for the HLOD actor to set it.

Regarding FISMComponentDescriptorBase, ah ok thanks. We had them locally added prior to that CL so perhaps there was confusing merge history on my end. Sorry for the confusion there.

Thanks,

Lucas

Ok, that’s fair.

Just to confirm though, we’re talking about IWorldPartitionHLODObject::SetVisibility not USceneComponent::SetVisibility which I agree would be a very risky change due to the other flags and their behavior. There is also FFastGeoComponentCluster::UpdateVisibility() which is another implementation of IWorldPartitionHLODObject so using FastGeo the component override method unfortunately doesn’t work. The FastGeo code uses what seems to be a unique codepath: UpdatePrimitivesDrawnInGame_RenderThread which also seems to assume bVisible has the semantics of bForceHidden since it removes non-visible objects from the Lumen scene which ignores bAffectIndirectLightingWhileHidden=true

For sure though, I understand, any change which alters existing behavior has some risk, but it does seem like a bug that bCastHiddenShadow objects are not removed from the scene when a HLOD is disabled and that the engine is not totally consistent here.

Thanks,

Lucas

Thanks! Appreciate you following up on this with those folks.

Yeah, FastGeo does complicate it since otherwise working around it in a custom component isn’t so bad.

Thanks! I’ll take a look, but I’m not sure if that would work for us. The problem is that later that code pushes updates to the actual scene via FScene::UpdatePrimitivesDrawnInGame_RenderThread and that special path (which seems to be only used by FastGeo) doesn’t consider the other flags, only bDrawnInGame