Crash generating navigation with new convex rasterization feature

Development Programming & Scripting
, ,
1

Exception Message

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0x00000229cc0cf9d0

Function

IstariServer!AddSpanInTempColumns::addSpan

Location

RecastRasterization.cpp(1113)

Not sure if it’s choking on a particular asset but it crashes on a thread so it’s hard to know what asset it would have even been processing.

I guess this isn’t a valid index. But it’s not a real array, it’s a struct with it’s own format I don’t fully understand so it may be better to have someone familiar with the feature to comment.

hf.tempSpanColumns[tempColumnIdx];
2

Hi there,

Can you verify that you have the following commit in your Engine revision history:

CL 37241451

I note the line number you report doesn’t seem to line up with the `hf.tempSpanColumns[tempColumnIdx];` line in the latest 5.6 branch (this is on line 1126 in the latest 5.6 commit). So maybe you are missing this commit. This commit adds better out of bounds checking to ensure that this line can’t access invalid memory.

The main relevant change is as follows:

// --- Replace ---
rcAssert(tempColumnx >= 0 && tempColumnx < ibwidth);
rcAssert(tempColumnz >= 0 && tempColumnz < ibheight);
// --- With ---
if (tempColumnx < 0 || tempColumnz < 0 || tempColumnx >= ibwidth || tempColumnz >= ibheight)
{
    // the span is outside of the verts bounding box, we have to skip it because it would mean we access invalid memory
    if (ctx)
    {
        ctx->log(RC_LOG_WARNING, "rcRasterizeTriangles trying to rasterize a triangle as filled convex but some spans are outside of the provided bounding box. Those spans will be skipped.");
    }
    return;
}

Regards,

Lance Chaney.