How to properly integrate Libtorch (Pytorch) and Unreal Engine 4?

Visual Studio 2019
Unreal Engine 4 (4.26.2)
Libtorch 1.10 versión CPU release mode

When I integrate Libtorch (Pytorch) into Visual studio 2019 in release mode. I can run my algorithms without problems.

However, when I integrate Libtorch into Unreal Engine 4 in Debelopment Editor mode I get an error in runtime in the funtion at :: Tensor output = module.forward (inputs) .toTensor ();

I have tried for a couple of weeks to resolve the error at runtime, but I have not succeeded.

Please, could someone give me any suggestions?

In order to replicate the error:

  • Create a new Project in PyCharm
  • Install PyTorch: pip3 install torch torchvision torchaudio
    -Write the following code:
 import torch
 import torchvision
 # An instance of your model.
 model = torchvision.models.resnet18()
 # An example input you would normally provide to your model's forward() method.
 example = torch.rand(1, 3, 224, 224)
 # Use torch.jit.trace to generate a torch.jit.ScriptModule via tracing.
 traced_script_module = torch.jit.trace(model, example)"D:\\")

-Download Libtorch: Stable (1.10), Windows, LibTorch, C++/Java and CPU (release mode).

-Extract the zip in your PATH.

-Create a project in Unreal Engine 4 named MyProject of type ThirdPerson and of type C++ (the default project is in mode Development Editor).

-Close the editor of UE4

-Open the properties in Project → MyProject Properties. Select the configuration Development_Editor and VC++ Directories.

  • Add the paths in External Include Directories:

  • Add the library int Library Directories:

  • Add PublicAdditionalLibraries int the MyProject.Build.cs:

public class MyProject : ModuleRules
	public MyProject(ReadOnlyTargetRules Target) : base(Target)
		PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;
		PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "InputCore", "HeadMountedDisplay" });
  • Paste the dynamic libraries (dlls) in the Win64 directory of the UE4 project: (PATH)\MyProject\Binaries\Win64
    Librarys: asmjit.dll, c10.dll. fbgemm.dll, libiomp5md.dll, libiompstubs5md.dll, torch.dll, torch_cpu.dll and uv.dll

  • Paste on the top of file MyProjectCharacter.h: #include <torch/script.h>. If you Build the project you get a lot of notes, warnings and errors.

  • Replace :#include <torch/script.h> with:

#pragma warning( push )
#pragma warning(disable: 4582)
#pragma warning(disable: 4583)
#pragma warning(disable: 4458)
#pragma warning(disable: 4624)
#pragma warning(disable: 4800)
#pragma warning(disable: 4541)
#include <torch/script.h> // One-stop header.
#pragma warning( pop )
  • Add the follow code in the constructor of your MyProjectCharacter.cpp
torch::jit::script::Module module = torch::jit::load("D:\\");
assert(module != nullptr);
std::cout << "ok\n";
  • Build the project, everything should run fine.

  • Add the following code after the code above:

// Create a vector of inputs.
std::vector<torch::jit::IValue> inputs;
inputs.push_back(torch::ones({1, 3, 224, 224}));

// Execute the model and turn its output into a tensor.
at::Tensor output = module.forward(inputs).toTensor();

std::cout << output.slice(/*dim=*/1, /*start=*/0, /*end=*/5) << '\n';
  • Select Debug  Options and uncheck: Use the new Exception Helper

  • Build and Run, then, you get an error in runt time in: at::Tensor output = module.forward(inputs).toTensor();

  • If you continue you get a new runtime error:
    Unhandled exception at 0x00007FFE8618F199 (ntdll.dll) in UE4Editor.exe: 0xC0000374: A heap has been corrupted (parameters: 0x00007FFE861F77F0).

-If you continue your project your project will run normally. (for example: Question Answering: BERT(Bidirectional Encoder Representation with Transformers) with LibTorch & UE4 - YouTube)

  • I have tried for a couple of weeks to resolve the error at runtime (for example, disabling optimization with #pragma optimize ("", off)), but I have not succeeded. The error in the heap continues.