Engine bug related to HISM / Foliage Spawners causing map/tile corruption

Vaei · October 5, 2018, 11:00am

Hi guys, this one has really been wrecking our artist’s progress and the code is really strange, been hard to decipher and harder to actually fix.

What happens is that any one of our artists will be working away, there is no problem with the tile, then at some point the crash starts occurring, and if it’s pushed we all start crashing when loading the tile that’s corrupted.

So the artist has to revert, losing tons of work. Over and over again. We have been unable to find how to reproduce it.

We don’t have the luxury of sitting idle while epic eventually fixes it.

First, this is the crash:


Assertion failed: (void*)((&ElementData[InstanceIndex]) + 1) <= (void*)(InstanceOriginDataPtr + CurrentSize) [File:C:\UnrealEngineProgram\ROS4.20.2\UnrealEngine\Engine\Source\Runtime\Engine\Public\StaticMeshResources.h] [Line: 1231]



0x00007ffce16ca388 KERNELBASE.dll!UnknownFunction ]
0x00007ffcb7385f60 UE4Editor-ApplicationCore.dll!FWindowsErrorOutputDevice::Serialize() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\applicationcore\private\windows\windowserroroutputdevice.cpp:65]
0x00007ffc7b227f0c UE4Editor-Core.dll!FOutputDevice::LogfImpl() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\core\private\misc\outputdevice.cpp:70]
0x00007ffc7b1b813b UE4Editor-Core.dll!FDebug::AssertFailed() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\core\private\misc\assertionmacros.cpp:425]
0x00007ffc786a984b UE4Editor-Engine.dll!UInstancedStaticMeshComponent::BuildRenderData() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\engine\private\instancedstaticmesh.cpp:1103]
0x00007ffc786eaae2 UE4Editor-Engine.dll!UInstancedStaticMeshComponent::InitPerInstanceRenderData() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\engine\private\instancedstaticmesh.cpp:2129]
0x00007ffc78657ef3 UE4Editor-Engine.dll!UHierarchicalInstancedStaticMeshComponent::OnPostLoadPerInstanceData() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\engine\private\hierarchicalinstancedstaticmesh.cpp:2838]
0x00007ffc7abfb83b UE4Editor-CoreUObject.dll!UObject::ConditionalPostLoad() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\coreuobject\private\uobject\obj.cpp:1011]
0x00007ffc7ace1881 UE4Editor-CoreUObject.dll!EndLoad() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\coreuobject\private\uobject\uobjectglobals.cpp:1630]
0x00007ffc7acfbe91 UE4Editor-CoreUObject.dll!LoadPackageInternal() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\coreuobject\private\uobject\uobjectglobals.cpp:1375]
0x00007ffc7acfb099 UE4Editor-CoreUObject.dll!LoadPackage() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\coreuobject\private\uobject\uobjectglobals.cpp:1471]
0x00007ffc61012d1d UE4Editor-WorldBrowser.dll!FWorldTileModel::LoadLevel() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\editor\worldbrowser\private	iles\worldtilemodel.cpp:595]
0x00007ffc610132df UE4Editor-WorldBrowser.dll!FLevelCollectionModel::LoadLevels() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\editor\worldbrowser\private\levelcollectionmodel.cpp:590]
0x00007ffc61014627 UE4Editor-WorldBrowser.dll!FLevelModel::MakeLevelCurrent() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\editor\worldbrowser\private\levelmodel.cpp:307]
0x00007ffc61038c17 UE4Editor-WorldBrowser.dll!SWorldTileItem::OnMouseButtonDoubleClick() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\editor\worldbrowser\private	iles\sworldtileitem.cpp:323]
0x00007ffc7781887a UE4Editor-Slate.dll!FEventRouter::Route<FReply,FEventRouter::FBubblePolicy,FPointerEvent,<lambda_eeb33fd1b480e3cad58a1531d90d2e14> >() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\slate\private\framework\application\slateapplication.cpp:268]
0x00007ffc778ad48e UE4Editor-Slate.dll!FSlateApplication::RoutePointerDoubleClickEvent() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\slate\private\framework\application\slateapplication.cpp:5922]
0x00007ffc77894a7a UE4Editor-Slate.dll!FSlateApplication::ProcessMouseButtonDoubleClickEvent() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\slate\private\framework\application\slateapplication.cpp:5909]
0x00007ffc7788858a UE4Editor-Slate.dll!FSlateApplication::OnMouseDoubleClick() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\slate\private\framework\application\slateapplication.cpp:5885]
0x00007ffcb737cadd UE4Editor-ApplicationCore.dll!FWindowsApplication::ProcessDeferredMessage() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\applicationcore\private\windows\windowsapplication.cpp:1740]
0x00007ffcb73700a0 UE4Editor-ApplicationCore.dll!FWindowsApplication::DeferMessage() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\applicationcore\private\windows\windowsapplication.cpp:2182]
0x00007ffcb737ec52 UE4Editor-ApplicationCore.dll!FWindowsApplication::ProcessMessage() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\applicationcore\private\windows\windowsapplication.cpp:1416]
0x00007ffcb736c149 UE4Editor-ApplicationCore.dll!FWindowsApplication::AppWndProc() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\applicationcore\private\windows\windowsapplication.cpp:732]
0x00007ffce3d06cc1 USER32.dll!UnknownFunction ]
0x00007ffce3d06693 USER32.dll!UnknownFunction ]
0x00007ffcb737fda6 UE4Editor-ApplicationCore.dll!FWindowsPlatformApplicationMisc::PumpMessages() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\applicationcore\private\windows\windowsplatformapplicationmisc.cpp:129]
0x00007ff75f2b54c3 UE4Editor.exe!FEngineLoop::Tick() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\launch\private\launchengineloop.cpp:3417]
0x00007ff75f2c4f9c UE4Editor.exe!GuardedMain() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\launch\private\launch.cpp:166]
0x00007ff75f2c501a UE4Editor.exe!GuardedMainWrapper() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\launch\private\windows\launchwindows.cpp:144]
0x00007ff75f2d30cc UE4Editor.exe!WinMain() [c:\unrealengineprogram\ros4.20.2\unrealengine\engine\source\runtime\launch\private\windows\launchwindows.cpp:223]
0x00007ff75f2d4db2 UE4Editor.exe!__scrt_common_main_seh() [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:288]
0x00007ffce3bd3034 KERNEL32.DLL!UnknownFunction ]
0x00007ffce45f1461 ntdll.dll!UnknownFunction ]

And on line 1231 it’s the first check from this function:


    FORCEINLINE_DEBUGGABLE void SetInstanceOriginInternal(int32 InstanceIndex, const FVector4& Origin) const
    {
        FVector4* ElementData = reinterpret_cast<FVector4*>(InstanceOriginDataPtr);
        uint32 CurrentSize = InstanceOriginData->Num() * InstanceOriginData->GetStride();
        check((void*)((&ElementData[InstanceIndex]) + 1) <= (void*)(InstanceOriginDataPtr + CurrentSize));
        check((void*)((&ElementData[InstanceIndex]) + 0) >= (void*)(InstanceOriginDataPtr));

        ElementData[InstanceIndex] = Origin;
    }


        check((void*)((&ElementData[InstanceIndex]) + 1) <= (void*)(InstanceOriginDataPtr + CurrentSize));

And if I comment it out there are several other identical checks it fails.

So here’s what I’ve worked out:
InstanceOriginDataPtr is a uint8*
The cast to an FVector4* reserves the space in memory
I believe they do this because they don’t know the data type at compile time
I believe the check is essentially mimicking an array out of bounds error, basically, it exceeds allocated memory?

I need help with:
Cementing what it actually does
Fixing it!

There’s also a thread on AnswerHub. This thread has no solution (no one really tried) and it spawned a bug report that has nothing to do with it! I fixed the bug in the report, but it’s unrelated.

Edits with new information:

Artist claims he spawned significantly more stuff in another tile that works fine, doesn’t seem like it has some unusual memory limitation

Vaei · October 6, 2018, 9:24am

We got the map to load with a band-aid fix:

Overwrite the following files, we’re on 4.20 and we overwrote from Master (4.22):

InstancedStaticMeshComponent.h
HierarchicalInstancedStaticMeshComponent.cpp
InstancedStaticMesh.h
InstancedStaticMesh.cpp
StaticMeshResources.h

Comment out each line with a check that starts like this from StaticMeshResources.h:


check((void*)((&ElementData[InstanceIndex]) + 1) <= // ...

Vaei · October 6, 2018, 10:14am

Now that we were able to load the map we noticed at varying angles the vertices on the trees extrude wildly. Basically the HISMs vertex data corrupted. It’s similar to when a GPU overheats (I have checked that other members of the team experience this, and they do, it’s just mentioned for visualization)like this

The issue is now to find what’s been corrupting it.

Manoel.Neto · October 8, 2018, 3:59pm

I recommend using RenderDoc to debug the rendering. There’s even a built in plugin for it. You can view the contents of the buffers and debug the shaders.

ZioYuri78 · October 9, 2018, 1:15pm

I’m the author of the answerhub thread mentioned above, now my new test crash in a new different place, it no longer fail the checks in SetInstanceOriginInternal(…) function but it crash here:



void FStaticMeshInstanceBuffer::UpdateFromCommandBuffer_RenderThread(FInstanceUpdateCmdBuffer& CmdBuffer)
{
    QUICK_SCOPE_CYCLE_COUNTER(STAT_FStaticMeshInstanceBuffer_UpdateFromCommandBuffer_RenderThread);

    int32 NumCommands = CmdBuffer.NumInlineCommands();
    int32 NumAdds = CmdBuffer.NumAdds;
    int32 AddIndex = INDEX_NONE;

    if (NumAdds > 0)
    {
        AddIndex = InstanceData->GetNumInstances();
        int32 NewNumInstances = NumAdds + InstanceData->GetNumInstances();
        InstanceData->AllocateInstances(NewNumInstances, GIsEditor ? EResizeBufferFlags::AllowSlackOnGrow | EResizeBufferFlags::AllowSlackOnReduce : EResizeBufferFlags::None, false); // In Editor always permit overallocation, to prevent too much realloc
    }

    for (int32 i = 0; i < NumCommands; ++i)
    {
        const auto& Cmd = CmdBuffer.Cmds*;
        switch (Cmd.Type)
        {
        case FInstanceUpdateCmdBuffer::Add:
            InstanceData->SetInstance(AddIndex++, Cmd.XForm, 0);
            break;

**case FInstanceUpdateCmdBuffer::Hide:**
**InstanceData->NullifyInstance(Cmd.InstanceIndex);**
**break;**

        case FInstanceUpdateCmdBuffer::Update:
            InstanceData->SetInstance(Cmd.InstanceIndex, Cmd.XForm, 0);
            break;
        case FInstanceUpdateCmdBuffer::EditorData:
            InstanceData->SetInstanceEditorData(Cmd.InstanceIndex, Cmd.HitProxyColor, Cmd.bSelected);
            break;
        case FInstanceUpdateCmdBuffer::LightmapData:
            InstanceData->SetInstanceLightMapData(Cmd.InstanceIndex, Cmd.LightmapUVBias, Cmd.ShadowmapUVBias);
            break;
        default:
            check(false);
        }
    }

    UpdateRHI();
}

Konflict · May 22, 2019, 4:57am

It may be an editor related crash only. Once i cook a build (tested both dev or shipping) out of it ain’t freak out.

The order of things seems to be important to get a crash in the editor. First you start with a hism with a null mesh. Next, you add an instance -> set the mesh -> then you can try to run any operation (add/update/clear) they will all fail.

I only tested this with blueprints, maybe you’d get different results doing this from code. Anyways, it’ll be better to wait for the mesh to be emplaced (account for async loading etc) before operating on a hism comp, just to be safe.

As for the trashed content, since you bypassed all checks, it’s possible you are merely operating on some old/invalid/un/partially initialized data, that sounds rather dangerous.