Skip Interchange Factory Nodes During Import Not Working

Development Pipeline & Plugins
,
1

I’m writing a c++ plugin that imports .nif files (NetImmerse, Gamebryo, Creation Engine) using Unreal’s Interchange system, but when I drag multiple nif assets at once into the editor, I’m getting a bunch of prompts to overwrite textures for each asset because many of them point to the same texture file.

Batch importing .nif files with shared textures triggers numerous “Overwrite Asset?” prompts because multiple files reference the same texture paths (e.g., architecture/farmhouse/wood01.dds). Pressing “Yes to all” only applies to all textures for a single mesh, not all meshes. I’m looking to import thousands of files and do these kinds of batch operations multiple times, and ship the plugin to other people, so this won’t do.

Unreal provides UInterchangeFactoryBaseNode::SetSkipNodeImport (lets me skip importing nodes). It seems like it was made exactly for my use case (see ShouldSkipNodeImport docs linked at the bottom as well):
"Nodes can be in a situation where we have to skip the import process because we cannot import the associated asset for multiple reasons. For example:

  • An asset can already exist and is being imported by another concurrent import task (such as a user importing multiple files at the same time in the same content folder)."

My approach was to filter factory nodes using a set of keys for filtering (populated when a factory starts work) and UInterchangeFactoryBaseNode::SetSkipNodeImport. The intent is for the first task to import a texture, claim its path (add key to the set), and subsequent tasks check the set and mark their factory nodes to skip.

However, I just couldn’t get it to skip anything. I get the prompts to override regardless.
Here’s a basic diagram of what I’m doing:
Say I have 4 meshes trying to import the texture in parallel. Here’s basic diagram that flows in time from left to right:

obj1--tex1fac---finished importing    (path claimed)
obj2---tex1fac---finished importing   (skipped texture creation)
obj3---tex1fac---finished importing   (skipped texture creation)
obj4----tex1fac---finished importing  (skipped texture creation)

However, even when SetSkipNodeImport is called, the override dialog windows still appear.

Log Snippet (Importing 2 files sharing 4 textures) (condensed):

// File 1 imports first and claims the texture paths
LogNifPipeline: DEBUG: ClaimTexturePath(C:/Users/drkuz/Downloads/meshes/textures/architecture/riften/RiftenCanalWood01.dds) - Path was NOT previously claimed
LogNifPipeline: Claiming texture path: C:/Users/drkuz/Downloads/meshes/textures/architecture/riften/RiftenCanalWood01.dds - Total claimed paths: 1

// File 2 correctly identifies the path as claimed and marks it to skip
LogNifPipeline: DEBUG: IsTexturePathClaimed(C:/Users/drkuz/Downloads/meshes/textures/architecture/riften/RiftenCanalWood01.dds) = TRUE
LogNifPipeline: DEBUG: ProcessTextureFactoryNodes - Path was claimed by ANOTHER task/import
LogNifPipeline: DEBUG: ProcessTextureFactoryNodes - SETTING NODE TO SKIP IMPORT
LogNifPipeline: Skipping import for claimed texture path: C:/Users/drkuz/Downloads/meshes/textures/architecture/riften/RiftenCanalWood01.dds

// But the import is still attempted, and the overwrite prompt still appears
LogNifPipeline: DEBUG: ProcessTextureFactoryNodes - Complete. Skipped: 4, Will Import: 0
LogNifPipeline: DEBUG: ExecutePipeline - Completed for file: C:/Users/drkuz/Downloads/meshes/dockcolstr02.nif - Claimed 0 textures
LogNifImport: Texture file found: C:/Users/drkuz/Downloads/meshes/textures/architecture/riften/RiftenCanalWood01_n.dds
LogNifImport: Texture file found: C:/Users/drkuz/Downloads/meshes/textures/architecture/riften/RiftenCanalWood01.dds
LogNifImport: Texture file found: C:/Users/drkuz/Downloads/meshes/textures/architecture/solitude/SGrate.dds
LogNifImport: Texture file found: C:/Users/drkuz/Downloads/meshes/textures/architecture/solitude/SGrate_n.dds
LogDDSImport: DDS format supported
LogDDSImport: *** GetTexturePayloadData called for: C:/Users/drkuz/Downloads/meshes/textures/architecture/riften/RiftenCanalWood01.dds ***
LogDDSImport: *** GetTexturePayloadData called for: C:/Users/drkuz/Downloads/meshes/textures/architecture/solitude/SGrate_n.dds ***
LogDDSImport: *** GetTexturePayloadData called for: C:/Users/drkuz/Downloads/meshes/textures/architecture/solitude/SGrate.dds ***
LogDDSImport: *** GetTexturePayloadData called for: C:/Users/drkuz/Downloads/meshes/textures/architecture/riften/RiftenCanalWood01_n.dds ***
LogDDSImport: Detected normal map from filename: C:/Users/drkuz/Downloads/meshes/textures/architecture/solitude/SGrate_n.dds
LogDDSImport: Detected normal map from filename: C:/Users/drkuz/Downloads/meshes/textures/architecture/riften/RiftenCanalWood01_n.dds
LogSlate: Window 'Message' being destroyed
Message dialog closed, result: Yes, title: Message, text: Are you sure you want to override asset 'Texture2D /Game/Test/T_RiftenCanalWood01.T_RiftenCanalWood01'?
LogSlate: Window 'Message' being destroyed
Message dialog closed, result: Yes, title: Message, text: Are you sure you want to override asset 'Texture2D /Game/Test/T_RiftenCanalWood01_n.T_RiftenCanalWood01_n'?
LogSlate: Window 'Message' being destroyed
Message dialog closed, result: Yes, title: Message, text: Are you sure you want to override asset 'Texture2D /Game/Test/T_SGrate.T_SGrate'?
LogSlate: Window 'Message' being destroyed
Message dialog closed, result: Yes, title: Message, text: Are you sure you want to override asset 'Texture2D /Game/Test/T_SGrate_n.T_SGrate_n'?

My Pipeline Code (Setting Skip Flag):

  // In ProcessTextureFactoryNodes:
  if (FactoryNode)
  {
    // If the path is already claimed by ANOTHER import task (not this one)
    // or we found an existing texture, skip the import process for this node
    if (bPathAlreadyClaimed || ExistingTexture)
    {
      UE_LOG(LogNifPipeline, Log, TEXT("DEBUG: ProcessTextureFactoryNodes - SETTING NODE TO SKIP IMPORT"));
      FactoryNode->SetSkipNodeImport();
      NodesSkipped++;
  
      if (ExistingTexture)
      {
        // If we have a valid texture reference, point to it
        FactoryNode->SetCustomReferenceObject(FSoftObjectPath(ExistingTexture));
        UE_LOG(LogNifPipeline, Log, TEXT("Reusing existing texture for %s: %s"),
               *TexturePath, *ExistingTexture->GetPathName());
      }
      else
      {
        UE_LOG(LogNifPipeline, Log, TEXT("Skipping import for claimed texture path: %s"), *TexturePath);
      }
    }
  }

Plugin+Project+Assets+NifPipeline.cpp+Log.txt - Drive folder link (UE5.4 Windows only)

It seems like I’m correctly skipping nodes, but the system is showing prompts before it considers the skip flags (unless I’m misunderstanding or have done something wrong).
Why isn’t SetSkipNodeImport preventing the prompts as expected?

2

So, I did a little digging into the Unreal source code, and found that my suspicion was correct. The system is showing the dialog prompts just before analyzing the set skip attributes.
Code found in: Engine\Source\Runtime\Interchange\Engine\Private\Tasks\InterchangeTaskImportObject.cpp

I have no idea why that’s the case (it sounds very counterintuitive to me). Maybe someone with knowledge about the system like @UE_FlavienP could chime in?

So what’s the recommended solution? I haven’t been able to find any docs about filtering/removing factory nodes themselves (I’ve only seen stuff about this “skip” flag).

I don’t want to skip creating translate nodes for textures because I need all the materials to link up to the textures. And I can’t seem to reuse translate nodes across import tasks.

I considered filtering payload key assignment for texture translator nodes in the same way as I’m filtering factory nodes’ skip attributes (so they never try to import the texture multiple times in the first place), but then I believe the material parameters would just be pointers to nodes with no data references and so the materials wouldn’t get any textures assigned.

3

Edit: Testing more thoroughly the next day, it turns out this solution ultimately causes race conditions; see below.

Gosh, I feel really dumb because I’m just talking to myself now, but I finally figured it out. I swear I spent 4 days trying to figure out this issue before I posted—it wasn’t a preemptive post :sweat_smile:

Hopefully it will be helpful for others in the future, at least.

Here’s what I did that worked in the end, based on some suggestions:

  1. Filter texture translate nodes (keep only first unique instance).
  2. Store the texture path as a string attribute on the material node.
  3. In ExecutePostImportPipeline, retrieve the path via the string attribute, find the UTexture2D, and manually link it.
// Find the texture asset
    UTexture2D* Texture = FindTextureAsset(TexturePath);
    if (Texture)
    {
      // Set the material parameter - this connects the texture to the material
      FName ParamFName(*ParamName);
      Material->SetTextureParameterValueEditorOnly(ParamFName, Texture);
      UE_LOG(LogNifPipeline, Log, TEXT("Set material parameter %s to texture %s"), 
             *ParamName, *Texture->GetPathName());
    }

It’s a workaround and it feels like this defeats the whole purpose of translate nodes, but it works.

I’m still very confused as to why the prompts to overwrite appear before checking if the object should be skipped via SetSkipNodeImport. That makes no sense to me and I’d still like to know the reasoning behind that. It seems like a fundamental flaw in the parallel import logic if the provided mechanism to skip imports doesn’t prevent the overwrite dialog triggered by those very imports. Maybe that could be fixed in later engine versions if it wasn’t intentional?

Hopefully, we don’t have to do this workaround in the future…

1 Like
4

I saw your message and I am not well versed enough in the underlying code to tell you what is best to do. I am trying to get some dev eyes on your question. It is important for us that people can make their own translator and pipeline for a new format.

1 Like
5

Thanks!
I’ve got a new problem due to my workaround - race conditions.

Since I’m not linking every object with texture translate nodes, there is nothing telling the engine to “wait” for the textures to be imported. When importing just a few objects in parallel, the textures seem to be resolved quick enough that it’s not an issue. But just a few more objects and suddenly everything is breaking again - materials can’t find the textures because they don’t exist yet.

So I’m still just as stuck.

Doomed if I add texture translate nodes, doomed if I don’t :confused:

Edit 2:
Okay, I managed to get it working. The issue was that the DisplayNames for translate nodes weren’t unique (I thought only UIDs had to be unique but I guess not). The engine checks the DisplayName instead of UID for duplicates before the pipelines run (they check immediately after translate nodes are set up when the import task gets created). As a solution, I just created a unique hash for each object’s translate nodes and appended it to the original names, and the override prompt issue was fixed.

uint32 PathHash = GetTypeHash(SourceFilePath);

FString UniqueTextureAssetName = FString::Printf(TEXT("T_%s_%u"), *TextureBaseName, PathHash);

TextureNode->InitializeNode(TextureNodeUID, UniqueTextureAssetName, EInterchangeNodeContainerType::TranslatedAsset);

Then in the ExecutePipeline function I just removed the hash for the factory nodes so the final texture in the content browser didn’t have the hash:

FString RemoveNumericHashSuffix(const FString &DisplayLabel)
{
  int32 LastUnderscoreIndex = DisplayLabel.Find(TEXT("_"), ESearchCase::CaseSensitive, ESearchDir::FromEnd);
  if (LastUnderscoreIndex != INDEX_NONE)
  {
    return DisplayLabel.Left(LastUnderscoreIndex);
  }
  return DisplayLabel;
}

FString CleanName = RemoveNumericHashSuffix(DisplayLabel);
FactoryNode->SetAssetName(CleanName);

I kept the static map of already seen textures in the pipeline and told the factories to skip duplicate nodes:

if (ProcessedTexturePaths.Contains(TexturePath))
{
  FactoryNode->SetSkipNodeImport();
}

I also added some checks for existing textures that were already imported and not a part of my map.

I also added this function (that you can override from the default pipeline class) to reset the static map each time you do a batch of imports (it runs just once before the import dialog that lets users configure import options):

void UNifPipeline::PreDialogCleanup(const FName PipelineStackName)
{
  ProcessedTexturePaths.Empty();
}

I still set the material parameters in the ExecutePostFactoryPipeline function, but importantly, I used AddFactoryDependencyUID() in the ExecutePipeline stage to make the material nodes dependent on the texture translate nodes from factories that I didn’t filter out (to make sure the materials don’t get created until after the textures they reference, to avoid the race conditions):

// Get the texture translate node UID from the texture factory node.
FString TextureTranslateNodeUID;
if (!TextureFactoryNode->GetCustomTranslatedTextureNodeUid(TextureTranslateNodeUID))
  return false;

// Add the dependency directly
FString WinningFactoryNodeUID = UInterchangeFactoryBaseNode::BuildFactoryNodeUid(TextureTranslateNodeUID);
MatNode->AddFactoryDependencyUid(WinningFactoryNodeUID);

UE_LOG(LogNifPipeline, Log, TEXT("Added dependency: Material %s depends on texture %s"), *MatNode->GetUniqueID(), *WinningFactoryNodeUID);

Ideally, I’d rather be linking the parameter nodes directly instead of acting on the physical slots and textures post-factory, but I haven’t been able to figure out a way to link the texture parameter nodes in the pipeline phase yet. Post-factory assignment works well enough for now.

I hope this is helpful anyone in the future who’s trying to make a custom importer that’s actually useful.