Foliage Instance Misalignment and Dirty InstancedFoliageActors After World Load

Development World Creation
, ,
1

Hi,

I encountered an issue while working with InstancedFoliageActor in Unreal Engine 5.6.1.​

[Situation]

This issue does not occur in Landscape or Foliage Edit mode.

It happens after the level (with World Partition) is loaded and Waiting for texture resource to be ready for landscape

​[Issue]

When reading data from a Landscape HeightMap (via FLandscapeEditLayerComponentReadbackResult), ULandscapeHeightfieldCollisionComponent::SnapFoliageInstances is called

However, the ULandscapeHeightfieldCollisionComponent objects (corresponding to the Readback) and the foliage mesh instances become continuously misaligned.

As a result, the InstancedFoliageActor packages remain dirty even after resaving

ULandscapeHeightfieldCollisionComponent::SnapFoliageInstances(const FBox& InInstanceBox)
{
	UWorld* ComponentWorld = GetWorld();
	for (TActorIterator<AInstancedFoliageActor> It(ComponentWorld); It; ++It)
	{
		AInstancedFoliageActor* IFA = *It;
...
						World->LineTraceMultiByObjectType(Results, Start, End, FCollisionObjectQueryParams(ECollisionChannel::ECC_Visibility), FCollisionQueryParams(SCENE_QUERY_STAT(FoliageSnapToLandscape), true));
 
						bool bFoundHit = false;
						for (const FHitResult& Hit : Results)
						{
							if (Hit.Component == this)
							{
								bFoundHit = true;
								if ((InstanceLocation - Hit.Location).SizeSquared() > KINDA_SMALL_NUMBER)
								{
									IFA->Modify();
...
}

[Example values]

Instance.Location = {X=-126063.43248945594, Y=-10394.36923343333, Z=-4634.915273624938}

Hit.Location = {X=-126063.43248945594, Y=-10394.36923343333, Z=-4635.4142105467618}

(InstanceLocation - Hit.Location).SizeSquared() ≈ 0.24894

KINDA_SMALL_NUMBER = 1.e-4f ≈ 0.0001f

Among 8,542 foliage instances, about 89 (≈1%) exceed a distance of 1 unit, resulting in constant dirty states.

[​Desired]

It seems necessary to adjust the tolerance value or avoid unnecessary instance updates when the positional difference is within an acceptable floating-point error range

Thank you in advance.

2

Hi,

We could add a tolerance value, either via a Console Variable or per-foliage type. But the physics query results as well as the landscape update values are supposed to be deterministic from one run to another, so it’s surprising that the difference would exceed 1.e-4f (i.e. KINDA_SMALL_NUMBER)

The most likely explanation is that landscape is not up-to-date and therefore, on load, when landscape is updated on the GPU on then readback to the CPU, there’s a change in the heightmap that is detected, which could be tiny, and would therefore not show visually, but would end up producing a different heightfield still, which could then lead to the foliage being misplaced.

Please make sure that all landscape is up-to-date (Build / Build Landscape, and then save all changed landscape proxies) when running that test. If you still see the problem after all landscape proxies are up-to-date, then it’s possible there’s a determinism issue with the way that the landscape is updated on the GPU. We’ve seen this in the past with certain video cards/drivers, were tiny precision issues are introduced, leading to “off-by-one” errors in the 16 bits height value. It could be the case, for example, if some users are running with a certain GPU and others with another. GPUs can be quite volatile when it comes to precision… In order to validate that this is the problem indeed, you can use the CVar landscape.DumpHeightmapDiff 1 and also, landscape.DumpDiffDetails 1 when opening the map (sometimes) or when entering landscape mode (i.e. when a landscape update is requested). This will produce a set of files in the Saved/LandscapeLayers under your project’s folder (e.g. C:\MyProject\LandscapeLayers\2025.04.15-05.39.51\MyMap\Landscape\Heightmaps) with, for each heightmap that has been changed, a .png file for the previous and a .png for the new values, as well as a .txt file detailing exactly the pixels that have been changed and by how much.

If you find out that, after all the landscape proxies have been re-saved and you update the landscape again (by entering landscape mode, or using the CVar landscape.EditLayersLocalMerge.Enable 2 (in UE5.6), or landscape.ForceLayersFullUpdate (in UE5.7), you get some files generated in that LandscapeLayers folder, then there’s likely a determinism issue with something in the list of landscape edit layers (for example, the Water brush has had such determinism in the past, leading to slight differences every update), in which case we would be glad to investigate (if you can send us a repro case), or it could also be due to the GPU and/or driver introducing some imprecision (which we’ve also seen in the past), as explained above. There’s a CVar to counteract this latter case : landscape.DirtyHeightmapHeightThreshold which you can set to a value > 0. This represents a threshold under which a heightmap will not be considered as having changed. For example, if you set it to 1, that means that any imprecision <= 1 (over 65635 values, since heightmaps are stored as 16 bits integers) will not be considered as an actual change and the CPU will ignore that value on readback and SnapFoliageInstances shouldn’t be called as a result.

Please let us know how it goes and if you find out anything interesting. If you have a repro case you could share, that is also something we could look into.

And if it turns out that there’s no problem with landscape and that we indeed need to add a threshold in the foliage snapping code, we will be happy to do so.

Cheers,

Jonathan

3

Hi again,

The usual way of debugging this type of issue is to disable the edit layers one by one (and eventually BP brushes if you use them), and then try to reproduce the issue by saving, then reloading the map and running the merge of landscape layers if it hasn’t run (using landscape.EditLayersLocalMerge.Enable 2). And then inspect the results, using landscape.DumpHeightmapDiff 1 and landscape.DumpDiffDetails 1 to confirm whether the issue occurs or not.

Doing this should allow you to pinpoint which edit layer or BP brush introduces the discrepancy and then you can focus on debugging this. You can also try to run with landscape.EditLayersLocalMerge.Enable 0 and see if the same problem occurs. There have been problems with the new merge method in the past, so it’s a possibility (please note that landscape.EditLayersLocalMerge.Enable 0 has been altogether removed in the UE5.7 release so we won’t investigate or fix any problem with the old method from now on, but it can still be useful information if you find out that something doesn’t work consistently with the new method). But we’ve fixed all issues we’ve identified with the new method (in UE5.7 and beyond).

Another way, if you’re comfortable with it, is to use RenderDoc to analyze the edit layers merge process and compare 2 occurrences. You can do this by

  1. installing RenderDoc,
  2. running the editor with the -attachrenderdoc command line option,
  3. set the CVar landscape.RenderCaptureLayersNextHeightmapDraws 1 (that will trigger a Render Doc capture automatically the next time the merge operation is executed : please note that this CVar’s value represents the number of captures you want to run, so you need to set it to 1 again, or a higher number, after a capture has been done, since it will be decremented every time)
  4. then loading the map and running the merge (using the landscape.EditLayersLocalMerge.Enable trick above, or just landscape.ForceLayersFullUpdate, if you run with UE5.7)

This should open RenderDoc with the captured (.rdc) file and then you can see the entire process running on the GPU.

Then repeat the process with another run, (still with landscape.DumpHeightmapDiff/landscape.DumpDiffDetails 1, to confirm the discrepancy) and inspect, in the 2 captures and for a pixel where there’s a difference, at which step the difference is introduced.

Since the whole process runs on the GPU, this is unfortunately the only reliable way to see what’s going on. Please note that running with the -d3d11 commandline argument, if it’s possible for you, is a way to make the debugging process easier with RenderDoc, since D3D12 captures tend to be a bit harder to inspect.

This type of issue can be due to many factors :

  1. A usual culprit is BP brushes. These need to work deterministically and since it’s controlled by BP, it can be tough to debug. If you are using custom BP brushes, please inspect them to try and track down anything where the result could change from one time to another (Water or Landmass brushes have had such issues in the past but they should work reliably now)
  2. Still related to BP brushes, a usual suspect is ODSC (On Demand Shader Compilation) that was introduced a while ago (UE5.1 if I remember correctly) : BP brushes use BP functions like DrawMaterialToRenderTarget in order to render the brush. These are using materials from the Material Editor and unfortunately, there’s no way, from BP, to ask the engine to compile these materials in advance. So if the merge runs and the material used by the brush is not “ready” (i.e. compiled), it will render using the default grey material and the result of the merge will be changed (usually, in a big way, so I doubt this is the issue in your case). Once the merge has run once, the materials have been requested to compile so this error will tend not to occur again on subsequent runs. The way to fix this is to declare any material that your BP brush uses by implementing the BP function GetRenderDependencies. This will tell the engine to compile the material and will not let the merge run until they are.
  3. In a similar fashion, BP brushes can end up pulling on textures (e.g. noise textures, masks, etc.) that are streamable. The result of the merge needs to be deterministic so we need to make sure that any texture being used in the process is fully streamed in, so that sampling it gives a consistent result every time the merge runs. Similar to ODSC materials above, the way to achieve this is to declare the textures being used by the BP brush in the GetRenderDependencies BP function. Here also, the engine will trigger the loading of all mips and will wait for this to happen before running the merge. This type of issue (along with the ODSC one) usually happens on the first run of the merge after load (i.e. the first time the materials or textures are requested to compile/stream in). Similar to the ODSC issue, this issue tends to occur right after loading, if the landscape merge runs while texture mips are still streaming in in the background. This is why I suggested earlier that you reload the map entirely after disabling a BP brush to try and pinpoint the issue.
  4. Issues on the borders can also happen sometimes, for example if there exists a discrepancy in the source data (i.e. in one or several of the edit layers) of 2 neighboring proxies, since they “share” pixels on their borders (meaning : they each have a copy of the border pixel and it must match 100% in order to get deterministic results). If the user only loads a single proxy (this is not possible if you are using BP brushes, though, since we force-load all landscape proxies in such a case, precisely to avoid potential determinism issues in the BP brush), we try to disable the sculpt/paint brush when reaching the border in order to prevent the situation (it becomes red) but a user could have possibly worked with multiple proxies loaded, modified the border (in that case, the brush would not be restricted from sculpting, since it’s now not modifying the border of what’s loaded), and then only saved or submitted one of the 2 proxies. In such a situation, when we run the landscape edit layers merge, we only pick the border from one of the 2 proxies in order to “solve the ambiguity” but this process is simply a “patch” and will still mean that the results are not deterministic and will depend on the set of proxies that have been loaded [Content removed] is an example of such a problem)
  5. We have recently found an edge case with the Splines layer, where such errors could occur if a spline segment ends exactly at the border of 2 proxies

If you are able to upgrade to UE5.7, it would be good to see if this still happens there.

I hope it helps. Please let us know if you need further help or have additional details (RenderDoc captures, or even a minimal project with some consistent repro steps)

Cheers,

Jonathan

4

Hi,

After enabling landscape.DumpDiffDetails 1 and landscape.DumpHeightmapDiff 1 to inspect the heightmap, I checked the output and saw the following in LandscapeComponent-Heightmap_0[mip0]_diff:

  • Max height diff (at X=17 Y=127) = 206 (0.314%)
  • Max normal diff (at X=120 Y=126) = 84 (32.941%)

Given these values, it doesn’t seem to be a GPU precision issue.

Also, looking at the pattern, most of the changes are happening near the borders (pixel row = 0, 1, or height - 1, height - 2).

I’m trying to track down which classes derived from ALandscapeBlueprintBrushBase, or any other systems that might be modifying the heightmap.

Do you have any tips on how to find the functions or code paths that affect the heightmap?

While investigating this, I also found the following article helpful, so I’m sharing it here as well: Landscape Dirty on Load

Thank you in advance.