This question was created in reference to: [Intermittent “Content is missing from cook” [Content removed]
So it’s been a while since I posted the question above but I wanted to give an update in case anyone else runs into this. I believe we managed to figure out the issue (at least for our situation) and we have a kind of hacky workaround to solve it.
First, to summarize what can cause a _GEN_VARIABLE object to be missing and referenced:
There is a parent actor blueprint which has a component added to it, as well as a child blueprint derived from this.
Both the parent and child blueprints contain a _GEN_VARIABLE object containing the respective blueprint’s settings for this component.
During cooking, any _GEN_VARIABLE objects which are identical to the ones from the parent blueprint will be stripped out since they are not needed. This is determined in the code by UInheritableComponentHandler::IsRecordNecessary.
Instances of a blueprint actor placed in the level will reference the _GEN_VARIABLE object from the most derived blueprint in the inheritance hierarchy where this object hasn’t been stripped out.
The issue occurs if the output of UInheritableComponentHandler::IsRecordNecessary is not consistent. So if the blueprint is loaded multiple times during the cook process and a component record is determined to be necessary the first time around, actors can add references to this component record. If the blueprint is reloaded later on and IsRecordNecessary now returns false, the component will be stripped from the cook and we will get the error.
Now to the interesting part, what is actually triggering the output of IsRecordNecessary to be non-deterministic:
We have a parent blueprint with a static mesh component using the assigned mesh’s collision settings (bUseDefaultCollision = true).
We have a child blueprint derived from this which doesn’t change any settings on the static mesh component.
Static mesh components with bUseDefaultCollision = true will update the collision settings in their BodyInstance property to match the collision settings of the mesh asset. Those values will be saved in the blueprint.
When the collision settings of the actual mesh asset change but only the child blueprint is resaved, it will have different values saved in its BodyInstance than the parent blueprint.
Usually this should be handled in UStaticMeshComponent::Serialize as there is a call to UpdateCollisionFromStaticMesh, HOWEVER this will not work as intended if the static mesh is still compiling or just not loaded yet (in our case it still had the RF_NeedLoad flag). Generally this is not an issue, since UpdateCollisionFromStaticMesh will then later be called again from UStaticMeshComponent::PostStaticMeshCompilation and assign the correct values.
The issue occurs if a blueprint is compiled between the Serialize and PostStaticMeshCompilation calls. In that case, the BodyInstance property will not yet have been updated with the correct values and UInheritableComponentHandler::IsRecordNecessary will determine that the record is necessary, since the value of BodyInstance is different from the parent blueprint. If any actors based on this blueprint are now cooked, they will reference the child blueprint’s _GEN_VARIABLE object.
When the blueprint is reloaded later on and the mesh is already in memory and compiled, the collision settings in the BodyInstance property will be updated correctly during the Serialize call, the child and parent records will match when the blueprint is compiled, and the child blueprint’s _GEN_VARIABLE object will be stripped out of the cook.
The hacky workaround we implemented for this is to change FComponentComparisonHelper::AreIdentical to skip any collision-related properties in the comparison of the BodyInstance property if the parent UStaticMeshComponent has bUseDefaultCollision set to true.
This solves the issue, but I feel like this is something that should be solved on the level of the UStaticMeshComponent or FBodyInstance. However it seemed more risky to make any changes there, so I’ll leave that to you guys
Create an actor blueprint (A) with a static mesh component and assign a static mesh to it. Make sure the component’s bUseDefaultCollision property is checked, then save the blueprint.
Create another blueprint (B) derived from A. Do not change any settings here, just save it.
Place an instance of B in a level and save that as well.
Change the collision settings on the static mesh (e.g. assign a different collision preset) and save it. Then reload and resave the child blueprint B (not the parent!).
Cook your project’s content.
Observe that you may sometimes get the “Content is missing from cook” error due to the actor in the level referencing the StaticMeshComponent_GEN_VARIABLE object from blueprint B.
[Attachment Removed]
For anyone interested, here’s our version of FComponentComparisonHelper with the workaround. You can just copy this over the version in InheritableComponentHandler.cpp to get a quick and dirty fix.
struct FComponentComparisonHelper
{
// GRIMLORE Start dlehn: Static mesh components with bUseDefaultCollision update their collision profiles on load
// or later if their assigned static mesh is still compiling. This can lead to differences in their BodyInstance
// properties, depending on the last state saved in the asset.
// To circumvent this, we must ignore any properties inside BodyInstance which are overwritten when
// bUseDefaultCollision is true.
// This only needs to be handled in the editor, since mesh compilation isn't a thing in cooked builds and we can
// assume that all properties have been updated during cooking. Also, blueprint compilation shouldn't even run
// outside the editor I think, so not entirely sure why parts of this code are not editor-only in the first place.
#if WITH_EDITOR
static bool UsesStaticMeshCollisionProfile(UObject* Object)
{
UStaticMeshComponent* StaticMeshComponent = Cast<UStaticMeshComponent>(Object);
return StaticMeshComponent
&& StaticMeshComponent->bUseDefaultCollision
&& StaticMeshComponent->SupportsDefaultCollision();
}
static bool AreBodyInstancesIdenticalIgnoringCollision(
const FBodyInstance& BodyInstanceA,
const FBodyInstance& BodyInstanceB)
{
static_assert(ENGINE_MAJOR_VERSION == 5 && ENGINE_MINOR_VERSION == 7,
"You just performed an engine upgrade! Please make sure this comparison function is still checking the "
"correct properties. See UStaticMeshComponent::UpdateCollisionFromStaticMesh/FBodyInstance::LoadProfileData.");
static const TSet<FName> PropertiesToIgnore{
TEXT("CollisionProfileName"),
TEXT("ObjectType"),
TEXT("CollisionEnabled"),
TEXT("CollisionResponses"),
};
for (FProperty* Prop = FBodyInstance::StaticStruct()->PropertyLink; Prop; Prop = Prop->PropertyLinkNext)
{
if (Prop->ShouldDuplicateValue() == false || PropertiesToIgnore.Contains(Prop->GetFName()))
{
continue;
}
for (int32 Idx = 0; Idx < Prop->ArrayDim; Idx++)
{
if (Prop->Identical_InContainer(&BodyInstanceA, &BodyInstanceB, Idx, PPF_DeepComparison) == false)
{
return false;
}
}
}
return true;
}
#endif
// GRIMLORE End
static bool AreIdentical(UObject* ObjectA, UObject* ObjectB)
{
if (!ObjectA || !ObjectB || (ObjectA->GetClass() != ObjectB->GetClass()))
{
return false;
}
// GRIMLORE Start dlehn: See comment above UsesStaticMeshCollisionProfile.
#if WITH_EDITOR
static const FName BodyInstanceName = GET_MEMBER_NAME_CHECKED(UStaticMeshComponent, BodyInstance);
// Note: We are deliberately only checking whether one of the objects uses a different collision profile and
// not checking whether the meshes are identical. If either the mesh or bUseDefaultCollision differ, we'll
// detect a mismatch in the regular property comparisons below anyway.
const bool NeedsCustomBodyInstanceComparison = UsesStaticMeshCollisionProfile(ObjectA);
#endif
// GRIMLORE End
bool Result = true;
for (FProperty* Prop = ObjectA->GetClass()->PropertyLink; Prop && Result; Prop = Prop->PropertyLinkNext)
{
bool bConsiderProperty = Prop->ShouldDuplicateValue(); //Should the property be compared at all?
if (bConsiderProperty)
{
// GRIMLORE Start dlehn: See comment above UsesStaticMeshCollisionProfile.
#if WITH_EDITOR
if (NeedsCustomBodyInstanceComparison && Prop->GetFName() == BodyInstanceName)
{
if (AreBodyInstancesIdenticalIgnoringCollision(
*Prop->ContainerPtrToValuePtr<FBodyInstance>(ObjectA),
*Prop->ContainerPtrToValuePtr<FBodyInstance>(ObjectB)) == false)
{
Result = false;
}
continue;
}
#endif
// GRIMLORE End
for (int32 Idx = 0; (Idx < Prop->ArrayDim) && Result; Idx++)
{
if (!Prop->Identical_InContainer(ObjectA, ObjectB, Idx, PPF_DeepComparison))
{
Result = false;
}
}
}
}
if (Result)
{
// Allow the component to compare its native/ intrinsic properties.
Result = ObjectA->AreNativePropertiesIdenticalTo(ObjectB);
}
return Result;
}
};
Great investigation, you’re correct on all counts.
The best idea I had to harden against this kind of error was to cache decisions around template necessity for the duration of the cook. If this sounds helpful I’ll get the reviewed and committed.
Would look something like this:
namespace UE::InheritableComponentHandler::Private
{
// The cooker loads, regenerates and discards a Blueprint many times in one process. Each regeneration
// re-evaluates whether an override template differs from its parent template, and that comparison reads
// component state that depends on which assets were loaded and compiled at the time, so the verdict can
// flip between loads. A package saved against the first verdict then references a template that the
// second verdict strips. Nothing on disk changes during a cook, so the first verdict for a template path
// stands for the rest of the process.
static UE::FMutex GCookRecordNecessityMutex;
static TMap<FSoftObjectPath, bool> GCookRecordNecessity;
}
bool UInheritableComponentHandler::IsRecordNecessaryMemoizedWhenCooking(const FComponentOverrideRecord& Record) const
{
using namespace UE::InheritableComponentHandler::Private;
if (!Record.ComponentTemplate || !IsRunningCookCommandlet())
{
return IsRecordNecessary(Record);
}
const FSoftObjectPath TemplatePath(Record.ComponentTemplate);
{
UE::TUniqueLock Lock(GCookRecordNecessityMutex);
if (const bool* RecordedNecessity = GCookRecordNecessity.Find(TemplatePath))
{
return *RecordedNecessity;
}
}
const bool bIsNecessary = IsRecordNecessary(Record);
UE::TUniqueLock Lock(GCookRecordNecessityMutex);
return GCookRecordNecessity.FindOrAdd(TemplatePath, bIsNecessary);
}
That solution would work, though I see two issues with it:
The first result (the one you cache) is actually the incorrect one. So it would include unnecessary objects in the cook.
The cook remains non-deterministic, as whether an object is included in the cook can still vary based on timing.
I mean to be fair in practice it’s probably good enough since it’s very much an edge case that doesn’t affect many objects, but still I’d prefer a proper deterministic solution. Ideally one that can be implemented on the level of the StaticMeshComponent, since that should make sure it behaves deterministically.
My suggestion would be to replace the current UObject::AreNativePropertiesIdenticalTo virtual function with something like a virtual “ArePropertiesIdentical” function whose default implementation does the regular per-property comparison that FComponentComparisonHelper::AreIdentical did so far. Native classes could then overload this to add additional checks which so far were in AreNativePropertiesIdenticalTo (so that separate function would no longer be needed) but it would ALSO allow classes to override the rest of the checks, so the StaticMeshComponent (which coincidentally is also so the only object in the engine which implemented AreNativePropertiesIdenticalTo so far) could skip the base implementation and manually skip comparing the problematic properties just as I did in my workaround.
I slept on it and it occurred to me it would not help multiprocess cook, either.
>but still I’d prefer a proper deterministic solution. Ideally one that can be implemented on the level of the StaticMeshComponent, since that should make sure it behaves deterministically.
Yep. New UObject virutals actually have a high memory cost in the editor so I wont add a new one, but I’ll come up with something. Expect another update next week.
