Which observations are allowed and/or expected to work as a conv2d subobservation?

YoloSwaggerson
Mon, 16 Mar 2026 08:18:19 +0000
Programming & Scripting

Hello. So one of the observations I have is a conv2d observation. Initially I tried to use a continuous observation as its subobservation because I thought it made the most sense. That schema fragment looks like this:

FConv2dObservationParams Conv2DParamsLocal = Config.Conv2dParams;
Conv2DParamsLocal.InputWidth = Columns;
Conv2DParamsLocal.InputHeight = Rows;
Conv2DParamsLocal.InChannels = Config.Grids.Num();
auto RaindropContinuousObservation = ULearningAgentsObservations::SpecifyContinuousObservation(InObservationSchema,
	Rows * Columns * Config.Grids.Num(), 1.f, Key_Observation_Surrounding_LIDAR_Raindrop_Continuous);
auto RaindropObservationsConvolved = ULearningAgentsObservations::SpecifyConv2dObservation(InObservationSchema,
	RaindropContinuousObservation, Conv2DParamsLocal, Key_Observation_Surrounding_LIDAR_Raindrop_Convolved);

It compiles and, from what I could tell, schema is ok: I managed to get Imitation Recording running, I recorded and saved some data, there were no errors or warnings in logs.
But then, when I tried to create a policy object (ULearningAgentsPolicy::MakePolicy), I’ve got a nullptr exception at NNERuntimeBasicCpuModel.cpp:5432.

FConv2dInstance::FConv2dInstance(const FConv2dLayer& InConv2dLayer) : Conv2dLayer(InConv2dLayer)
{
	SubLayerInstance = Conv2dLayer.SubLayer->MakeInstance();
}

void FConv2dInstance::SetMaxBatchSize(const uint32 MaxBatchSize)
{
	NNE_RUNTIME_BASIC_TRACE_SCOPE(NNE::RuntimeBasic::Private::FConv2dLayerInstance::SetMaxBatchSize);
->	SubLayerInstance->SetMaxBatchSize(MaxBatchSize); // this is line 5432
	SubLayerOutputBuffer.SetNumUninitialized(MaxBatchSize * Conv2dLayer.SubLayer->GetOutputSize(), EAllowShrinking::No);
}

The SubLayerInstance is nullptr and constructor above where this variable is assigned is never called (or at least I never had a breakpoint triggered there).
I haven’t manage to understand why does this happen yet, but I tried to replace the continuous observation with a static array of floats observation as conv2d subobservation aaand… it worked. Well, at least, creating policy object doesn’t crash anymore. So this:

FConv2dObservationParams Conv2DParamsLocal = Config.Conv2dParams;
Conv2DParamsLocal.InputWidth = Columns;
Conv2DParamsLocal.InputHeight = Rows;
Conv2DParamsLocal.InChannels = Config.Grids.Num();
auto RaindropItemObservation = ULearningAgentsObservations::SpecifyFloatObservation(InObservationSchema, 1.f, Key_Observation_Surrounding_LIDAR_Raindrop_Item);
auto RaindropArrayObservation = ULearningAgentsObservations::SpecifyStaticArrayObservation(InObservationSchema, RaindropItemObservation,
	Rows * Columns * Config.Grids.Num(), Key_Observation_Surrounding_LIDAR_Raindrop_Array);
auto RaindropObservationsConvolved = ULearningAgentsObservations::SpecifyConv2dObservation(InObservationSchema,
	RaindropArrayObservation, Conv2DParamsLocal, Key_Observation_Surrounding_LIDAR_Raindrop_Convolved);

fixes the crash, and when the policy object is created, SubLayerInstance has a valid object inside. However, I still never saw the constructor to trigger so idk where is this object is created .

So the question is - is it how things are expected to work and static array of floats is the only viable observation here, or am I doing something wrong or is anything wrong with the current state of LA/NNERuntimeBasicCpu plugin?

For context, I use stock UE 5.7.2. I’m also attaching 2 screenshots with stack traces from calling ULearningAgentsPolicy::MakePolicy down to the place where I’ve been getting nullptr exception. One screenshot is with continuous observation and the other is with static array.

1 post - 1 participant