Multiple crashes using VirtualTextureCollection + fixes

We make use of the new “Experimental” `VirtualTextureCollection` as it is exactly what we needed to write a simple and elegant multi-material landscape blending system.

However, because it is experimental, we have encountered a number of issues-- most of which we think we have fixed. We would like to share these issues+fixes with the epic staff and the community for feedback and assistance for anyone else trying to utilize this feature.

Unfortunately, due to the character limit, we have to split this into multiple comments. It is too time consuming to partition these properly into their own posts or add individual reproduction projects for each. We hope that despite the unorthodox format this will still be of utility for those interested.

The full markdown file is provided as an attachment for ease of viewing.

[Attachment Removed]

Steps to Reproduce

  • Create a VirtualTextureCollection asset
  • Use it in the various ways described below
    [Attachment Removed]

### 1. Virtual Texture Collection Asserts when Existing Virtual Texture is Added

### Issue:

We hit an assert in `FVirtualTextureProducerCollection::RegisterProducer`

```cpp

FVirtualTextureProducerHandle FVirtualTextureProducerCollection::RegisterProducer(FRHICommandListBase& RHICmdList, FVirtualTextureSystem* System, const FVTProducerDescription& InDesc, IVirtualTexture* InProducer)

{

const uint32 ProducerWidth = InDesc.BlockWidthInTiles * InDesc.WidthInBlocks * InDesc.TileSize;

const uint32 ProducerHeight = InDesc.BlockHeightInTiles * InDesc.HeightInBlocks * InDesc.TileSize;

check(ProducerWidth > 0u);

check(ProducerHeight > 0u);

-- here --> check(InDesc.MaxLevel <= FMath::CeilLogTwo(FMath::Max(ProducerWidth, ProducerHeight)));

```

when a texture added to the collection that is already a virtual texture, because the `MaxLevel` is always one higher than `FMath::CeilLogTwo(FMath::Max(ProducerWidth, ProducerHeight)))`. When a non-virtual texture asset is added to the collection, no assert or exception occurs..

### **Fix**:

The bug originates in `FVirtualTextureCollectionResource::InitRHI`, where virtual entries used `FVirtualTexture2DResource::GetNumMips()` to populate the collection producer’s `MaxLevel`, making it consistently +1 too large.

Fixed by computing `MaxLevel` from virtual textures as `GetNumMips() - 1`, which is the same convention used by VT textures themselves (Texture2D.cpp:1634)

```cpp

Texture2D.cpp:1634

const int32 MaxLevel = VTData->GetNumMips() - FirstMipToUse - 1;

```

### Full patch:

```cpp

[Content removed]8 @@

CreateIndexTable(ProducerData);

// The number of mips required

- uint32 MaxVirtualMipCount = 0;

+ // FIX: FVTProducerDescription::MaxLevel is a mip *index* (0..MaxLevel), not a mip count.

+ uint32 MaxVirtualMipLevel = 0;

VirtualUniforms.SetNumZeroed(Textures.Num());

[Content removed]9 @@

// Keep the producer handle for later queries

TextureEntry.VirtualProducerHandle = VirtualResource->GetProducerHandle();

- // Actual maximum mip count required by this collection

- MaxVirtualMipCount = FMath::Max(MaxVirtualMipCount, VirtualResource->GetNumMips());

+ // Actual maximum mip level required by this collection

+ const uint32 VirtualResourceMaxLevel = VirtualResource->GetNumMips() > 0u ? (VirtualResource->GetNumMips() - 1u) : 0u;

+ MaxVirtualMipLevel = FMath::Max(MaxVirtualMipLevel, VirtualResourceMaxLevel);

// To block count

TextureEntry.BlockCount.X = FMath::DivideAndRoundUp(VirtualResource->GetSizeX(), static_cast<uint32>(BuildSettings.TileSize));

[Content removed]10 @@

TextureEntry.PhysicalTexture = Textures[TextureIndex];

TextureEntry.Format = PhysicalResource->GetPixelFormat();

- // Actual maximum mip count required by this collection

+ // Actual maximum mip level required by this collection

const uint32 EntryMipLimit = FMath::CeilLogTwo(FMath::Max(PhysicalResource->GetSizeX(), PhysicalResource->GetSizeY()));

const uint32 EntryMipCount = FMath::Min<uint32>(EntryMipLimit, PhysicalResource->GetState().MaxNumLODs);

- MaxVirtualMipCount = FMath::Max<uint32>(MaxVirtualMipCount, EntryMipCount);

+ MaxVirtualMipLevel = FMath::Max<uint32>(MaxVirtualMipLevel, EntryMipCount);

// To block count

TextureEntry.BlockCount.X = FMath::DivideAndRoundUp(PhysicalResource->GetSizeX(), static_cast<uint32>(BuildSettings.TileSize));

[Content removed]7 @@

ProducerDesc.BlockWidthInTiles = ProducerData.BlockCount.X;

ProducerDesc.BlockHeightInTiles = ProducerData.BlockCount.Y;

ProducerDesc.DepthInTiles = 1u;

- ProducerDesc.MaxLevel = MaxVirtualMipCount;

+ ProducerDesc.MaxLevel = MaxVirtualMipLevel;

ProducerDesc.NumTextureLayers = 1;

ProducerDesc.NumPhysicalGroups = 1;

ProducerDesc.Priority = EVTProducerPriority::Normal;

```

[Attachment Removed]

### 2. Virtual Texture Collection Indexing Not Working (UE-351412)

Link: <Unreal Engine Issues and Bug Tracker (UE\-351412\)>

### Background:

This issue is implicitly referenced in this EPS post

[Content removed]

Unfortunately the “patch” the user refers to in the link on the *this* word is broken

> please make sure to apply the patch from *this* thread which fixes an issue with indexing.

So I don’t know what the actual “sanctioned” patch from epic is. Looking at latest from `ue5-main` indicates nothing relevant in the region where the bug occurs either.

<https://github.com/EpicGames/UnrealEngine/blob/8b6692936882c541bb823e716d0b539ac3a0800b/Engine/Source/Runtime/Engine/Private/Materials/HLSLMaterialTranslator.cpp\#L7864\>

<https://github.com/EpicGames/UnrealEngine/blob/8b6692936882c541bb823e716d0b539ac3a0800b/Engine/Source/Runtime/Engine/Private/Materials/HLSLMaterialTranslator.cpp\#L8521\>

### Issue:

In `FHLSLMaterialTranslator::TextureFromCollection`, we have:

```cpp

if (TextureCollectionExpression->IsVirtualCollection())

{

if (ResultTextureType != MCT_Texture2D)

{

return Errorf(TEXT(“Texture collection virtual sampling only supports 2d textures”));

}

FString Uniforms = GetParameterCode(AddInternalCodeChunk(

MCT_Unexposed, TEXT(“FIndirectVirtualTextureUniform”),

*FString::Printf(TEXT(“GetIndirectVirtualTextureUniform(%i)”),

   TextureCollectionExpression\-\>GetTextureCollectionTypePrefixIndex())

));

return AddInternalCodeChunk(

static_cast<EMaterialValueType>(ResultTextureType | MCT_TextureVirtual |

  MCT\_TextureCollection \| MCT\_Unexposed), TEXT("FIndirectVirtualTextureEntry"),

*FString::Printf(

TEXT(“LoadIndirectVirtualTexture(%s, %s, %s)”),

*GetParameterCode(TextureCollectionCodeIndex),

*Uniforms,

*GetParameterCode(IndexIntoCollectionCodeIndex)

),

TextureCollectionExpression

);

}

```

`TextureCollectionTypePrefixIndex` is intentionally assigned lazily in the translator when the texture collection parameter is actually “accessed” as HLSL (`AccessUniformExpression` path for `MCT_TextureCollection`). The relevant logic is in `HLSLMaterialTranslator.cpp:4489`:

Because this lazy assignment is not invoked at this point, the uninitialized value cascades into possibly UB downstream in the shaders which consume this value, likely clamping it to 0.

### **Fix**:

Ensure the collection’s type-prefix index is initialized before emitting any HLSL that depends on it (mirror what `TextureCollectionCount()` already does).

### Full Patch:

```cpp

[Content removed]19 @@

{

return Errorf(TEXT(“Texture collection virtual sampling only supports 2d textures”));

}

+

+ // Ensure the collection has a stable type prefix index before we emit any HLSL that relies on it.

+ // (TextureCollectionCount() already does this; TextureFromCollection() needs the same for virtual collections.)

+ AccessUniformExpression(TextureCollectionCodeIndex);

FString Uniforms = GetParameterCode(AddInternalCodeChunk(

MCT_Unexposed, TEXT(“FIndirectVirtualTextureUniform”),

*FString::Printf(TEXT(“GetIndirectVirtualTextureUniform(%i)”), TextureCollectionExpression->GetTextureCollectionTypePrefixIndex())

));

return AddInternalCodeChunk(

```

[Attachment Removed]

3. ### Intermingling virtual and non virtual TextureCollection causes crash

### Issue part 1:

When using both a `VirtualTextureCollection` and `TextureCollection` inside a material, a crash will occur in `RHIUniformBufferDataShared.h:53`

```cpp

case UBMT_RDG_TEXTURE:

{

53: FRDGTexture* RDGTexture = Reader.Read<FRDGTexture*>(Resource);

FRHITexture* Texture = RDGTexture ? RDGTexture->GetRHI() : nullptr;

return Texture ? Texture->GetDefaultBindlessHandle() : FRHIDescriptorHandle();

}

```

This means a `UBMT_TEXTURE` slot in the material uniform buffer contains garbage, so `Reader.Read<FRHITexture*>` returns an invalid pointer

The root cause is bad packing of the material uniform buffer when virtual texture collections are present:

```cpp

FShaderParametersMetadata* FUniformExpressionSet::CreateBufferStruct()

{

// Make sure FUniformExpressionSet::CreateDebugLayout() is in sync

TArray<FShaderParametersMetadata::FMember> Members;

uint32 NextMemberOffset = 0;

if (!UniformTextureCollectionParameters.IsEmpty())

{

uint32 BindlessCollectionCount = 0;

uint32 VirtualCollectionCount = 0;

CountTextureCollections(BindlessCollectionCount, VirtualCollectionCount);

// Bindless collections do not make use of uniforms

if (VirtualCollectionCount)

{

static_assert(sizeof(FUintVector4) * UE::HLSL::IndirectVirtualTextureUniformDWord4Count == sizeof(UE::HLSL::FIndirectVirtualTextureUniform), “Unexpected size”);

new(Members) FShaderParametersMetadata::FMember(TEXT(“TextureCollectionPackedUniforms”), TEXT(“FIndirectVirtualTextureUniform”), __LINE__, NextMemberOffset, UBMT_UINT32, EShaderPrecisionModifier::Float, 1, 4, UE::HLSL::IndirectVirtualTextureUniformDWord4Count * VirtualCollectionCount, NULL);

NextMemberOffset += sizeof(UE::HLSL::FIndirectVirtualTextureUniform) * UniformTextureCollectionParameters.Num();

}

}

```

`FUniformExpressionSet::CreateBufferStruct()` declares `TextureCollectionPackedUniforms` only for virtual collections, but the code advances `NextMemberOffset` as if every texture collection contributed packed uniforms.

The runtime writer confirms only virtual collections write packed uniforms:

```cpp

void FUniformExpressionSet::FillUniformBuffer(const FMaterialRenderContext& MaterialRenderContext, TConstArrayView<IAllocatedVirtualTexture*> AllocatedVTs, const FRHIUniformBufferLayout* UniformBufferLayout, uint8* TempBuffer, int TempBufferSize) const

{

using namespace UE::Shader;

check(IsInParallelRenderingThread());

if (UniformBufferLayout->ConstantBufferSize > 0)

{

// stat disabled by default due to low-value/high-frequency

//QUICK_SCOPE_CYCLE_COUNTER(STAT_FUniformExpressionSet_FillUniformBuffer);

void* BufferCursor = TempBuffer;

check(BufferCursor <= TempBuffer + TempBufferSize);

if (!UniformTextureCollectionParameters.IsEmpty())

{

// Bindless collections do not make use of uniforms

for (const FMaterialTextureCollectionParameterInfo& ParameterInfo : UniformTextureCollectionParameters)

{

if (!ParameterInfo.bIsVirtualCollection)

{

continue;

}

```

### Fix/Full Patch:

```cpp

[Content removed]7 @@

{

static_assert(sizeof(FUintVector4) * UE::HLSL::IndirectVirtualTextureUniformDWord4Count == sizeof(UE::HLSL::FIndirectVirtualTextureUniform), “Unexpected size”);

new(Members) FShaderParametersMetadata::FMember(TEXT(“TextureCollectionPackedUniforms”), TEXT(“FIndirectVirtualTextureUniform”), __LINE__, NextMemberOffset, UBMT_UINT32, EShaderPrecisionModifier::Float, 1, 4, UE::HLSL::IndirectVirtualTextureUniformDWord4Count * VirtualCollectionCount, NULL);

- NextMemberOffset += sizeof(UE::HLSL::FIndirectVirtualTextureUniform) * UniformTextureCollectionParameters.Num();

+ NextMemberOffset += sizeof(UE::HLSL::FIndirectVirtualTextureUniform) * VirtualCollectionCount;

}

}

```

[Attachment Removed]

### Issue part 2:

After fixing part 1, one will encounter these sorts of compile issues in a material:

```cpp

[SM6] D:\Subsolar\gad\Engine\Shaders\Private\MaterialTemplate.ush(5178,145): Shader FLumenCardPS, Permutation 0, VF FLocalVertexFactory:

/Engine/Generated/Material.ush:5178:145: error: use of undeclared identifier ‘Material’

float4 Local187 \= ProcessMaterialVirtualColorTextureLookup(TextureVirtualSample(Material.TextureCollectionPhysical\_1, GetMaterialSharedSampler(Material.TextureCollectionSampler\_1, View\_SharedBilinearAnisoClampedSampler), Local186, 0, VTUniform\_Unpack(Local163\.PackedUniform)));

                                                                        ^

[SM6] Shader debug info dumped to: “D:\Subsolar\subsolar-main\Saved\ShaderDebugInfo\PCD3D_SM6\M_SuperMaterial_53426f2a1ec3d00a\Default\FLocalVertexFactory\FLumenCardPS\0”

D:\Subsolar\gad\Engine\Shaders\Private\MaterialTemplate.ush(5156,30): Shader FLumenCardPS, Permutation 0, VF FLocalVertexFactory:

/Engine/Generated/Material.ush:5156:30: error: use of undeclared identifier ‘Material’

Texture2D\<uint4\> Local165 \= Material.TextureCollectionPageTable\_1;

              ^

```

The origin of this issue is in `FHLSLMaterialTranslator::TextureSample` (`HLSLMaterialTranslator.cpp:7554/HLSLMaterialTranslator.cpp:7962`)

```cpp

else if (bFromCollection && bVirtualTexture)

{

CoerceParameter(TextureIndex, TextureType);

FMaterialUniformExpression* UniformExpression = GetParameterUniformExpression(TextureIndex);

if (!UniformExpression || !UniformExpression->GetTextureCollectionUniformExpression())

{

return Errorf(TEXT(“The provided uniform expression is not from a texture collection”));

}

FMaterialUniformExpressionTextureCollection* CollectionExpression = UniformExpression->GetTextureCollectionUniformExpression();

VirtualTextureIndex = CollectionExpression->GetTextureCollectionTypePrefixIndex();

if (SamplerSource != SSM_FromTextureAsset)

{

const bool bUseAnisoSampler = VirtualTextureScalability::IsAnisotropicFilteringEnabled() && MipValueMode != TMVM_MipLevel;

const TCHAR* SharedSamplerName = bUseAnisoSampler ? TEXT(“View.SharedBilinearAnisoClampedSampler”) : TEXT(“View.SharedBilinearClampedSampler”);

TextureName += FString::Printf(TEXT(“Material.TextureCollectionPhysical_%d, GetMaterialSharedSampler(Material.TextureCollectionSampler_%d, %s)”), VirtualTextureIndex, VirtualTextureIndex, SharedSamplerName);

}

else

{

TextureName += FString::Printf(TEXT(“Material.TextureCollectionPhysical_%d, Material.TextureCollectionSampler_%d”), VirtualTextureIndex, VirtualTextureIndex);

}

NumVtSamples++;

}

else if (bFromCollection)

{

FString TextureCode = GetParameterCode(TextureIndex);

SampleInfo.PageTableIndex = AddInternalCodeChunk(

MCT_Unexposed, TEXT(“Texture2D<uint4>”),

*FString::Printf(TEXT(“Material.TextureCollectionPageTable_%i”), VirtualTextureIndex)

);

FString Uniforms = GetParameterCode(AddInternalCodeChunk(

MCT_Unexposed, TEXT(“FIndirectVirtualTextureUniform”),

*FString::Printf(TEXT(“GetIndirectVirtualTextureUniform(%i)”), VirtualTextureIndex)

));

// Manually unpack the page table to redirect the page boundaries

SampleInfo.PageTableUniformIndex = AddInternalCodeChunk(

MCT_Unexposed, TEXT(“VTPageTableUniform”),

*FString::Printf(

TEXT(“UnpackIndirectVirtualTexturePageTableUniform(%s, %s.PackedPageTableUniform[0], %s.PackedPageTableUniform[1])”),

*TextureCode, *Uniforms, *Uniforms

)

);

VTPackedUniformName = FString::Printf(TEXT(“VTUniform_Unpack(%s.PackedUniform)”), *Uniforms);

VTStackIndex = AcquireVTStackIndex(SampleInfo, UV_Value, UV_Ddx, UV_Ddy);

VTPageTableLayerIndex = 0;

}

```

Basically there are two distinct indices in play:

* Collection uniform slot index (overall texture collection parameter index):

This is the index used to name/declare UB members like:

`Material.TextureCollection_#`, and for virtual collections: `Material.TextureCollectionPageTable_#`, `Material.TextureCollectionPhysical_#`, `Material.TextureCollectionSampler_#`. These are allocated in order of `UniformTextureCollectionParameters` (`MaterialUniformExpressions.cpp:611`).

* Virtual-collection type-prefix index:

This is the index into the packed-uniform array `Material.TextureCollectionPackedUniforms[…]`, which is sized by virtual collection count only (`MaterialUniformExpressions.cpp:409`).

The code uses `VirtualTextureIndex = TextureCollectionTypePrefixIndex;`to name `TextureCollectionPageTable/Physical/Sampler_`*,* but those UB members are indexed by the collection uniform slot (the overall `TextureCollection_` index). With mixed virtual + non-virtual collections, `prefix != slot`.

[Attachment Removed]

### Fix:

We now respects this separation:

* In VT collection sampling, it uses:

* Prefix index for `GetIndirectVirtualTextureUniform(prefix)` and thus `VTUniform_Unpack(…)`.

* Slot index for `TextureCollectionPageTable/Physical/Sampler` member names.

### Full Patch:

```cpp

[Content removed]29 @@

}

FMaterialUniformExpressionTextureCollection* CollectionExpression = UniformExpression->GetTextureCollectionUniformExpression();

- VirtualTextureIndex = CollectionExpression->GetTextureCollectionTypePrefixIndex();

+ const int32 TypePrefixIndex = CollectionExpression->GetTextureCollectionTypePrefixIndex();

+ if (TypePrefixIndex < 0)

+ {

+ return Errorf(TEXT(“Invalid texture collection type prefix index”));

+ }

+ const int32 TextureCollectionInputIndex = UniformTextureCollectionExpressions.Find(CollectionExpression);

+ if (TextureCollectionInputIndex == INDEX_NONE)

+ {

+ return Errorf(TEXT(“Unable to find texture collection uniform index.”));

+ }

+ // The VT packed uniforms are indexed by the virtual-collection prefix index, but the collection’s

+ // page table / physical texture / sampler are indexed by the collection uniform slot.

+ VirtualTextureIndex = TypePrefixIndex;

if (SamplerSource != SSM_FromTextureAsset)

{

const bool bUseAnisoSampler = VirtualTextureScalability::IsAnisotropicFilteringEnabled() && MipValueMode != TMVM_MipLevel;

const TCHAR* SharedSamplerName = bUseAnisoSampler ? TEXT(“View.SharedBilinearAnisoClampedSampler”) : TEXT(“View.SharedBilinearClampedSampler”);

- TextureName += FString::Printf(TEXT(“Material.TextureCollectionPhysical_%d, GetMaterialSharedSampler(Material.TextureCollectionSampler_%d, %s)”), VirtualTextureIndex, VirtualTextureIndex, SharedSamplerName);

+ TextureName += FString::Printf(TEXT(“Material.TextureCollectionPhysical_%d, GetMaterialSharedSampler(Material.TextureCollectionSampler_%d, %s)”), TextureCollectionInputIndex, TextureCollectionInputIndex, SharedSamplerName);

}

else

{

- TextureName += FString::Printf(TEXT(“Material.TextureCollectionPhysical_%d, Material.TextureCollectionSampler_%d”), VirtualTextureIndex, VirtualTextureIndex);

+ TextureName += FString::Printf(TEXT(“Material.TextureCollectionPhysical_%d, Material.TextureCollectionSampler_%d”), TextureCollectionInputIndex, TextureCollectionInputIndex);

}

NumVtSamples++;

[Content removed]23 @@

else if (bFromCollection)

{

FString TextureCode = GetParameterCode(TextureIndex);

+

+ FMaterialUniformExpression* UniformExpression = GetParameterUniformExpression(TextureIndex);

+ FMaterialUniformExpressionTextureCollection* CollectionExpression = UniformExpression ? UniformExpression->GetTextureCollectionUniformExpression() : nullptr;

+ if (!CollectionExpression)

+ {

+ return Errorf(TEXT(“The provided uniform expression is not from a texture collection”));

+ }

+

+ const int32 TextureCollectionInputIndex = UniformTextureCollectionExpressions.Find(CollectionExpression);

+ if (TextureCollectionInputIndex == INDEX_NONE)

+ {

+ return Errorf(TEXT(“Unable to find texture collection uniform index.”));

+ }

SampleInfo.PageTableIndex = AddInternalCodeChunk(

MCT_Unexposed, TEXT(“Texture2D<uint4>”),

- *FString::Printf(TEXT(“Material.TextureCollectionPageTable_%i”), VirtualTextureIndex)

+ *FString::Printf(TEXT(“Material.TextureCollectionPageTable_%i”), TextureCollectionInputIndex)

);

FString Uniforms = GetParameterCode(AddInternalCodeChunk(

```

[Attachment Removed]

Hi Will,

We appreciate you taking the time to contribute some fixes to the Virtual Texture Collection system. To properly review your changes, please submit them via a pull request through GitHub. That is our preferred method for examining and taking in external code contributions. For a detailed guide on how to do this, you can read up more here: https://dev.epicgames.com/documentation/en\-us/unreal\-engine/contributing\-to\-the\-unreal\-engine

[Attachment Removed]