Visual Studio 2019
Unreal Engine 4 (4.26.2)
Libtorch 1.10 versión CPU release mode
When I integrate Libtorch (Pytorch) into Visual studio 2019 in release mode. I can run my algorithms without problems.
However, when I integrate Libtorch into Unreal Engine 4 in Debelopment Editor mode I get an error in runtime in the funtion at :: Tensor output = module.forward (inputs) .toTensor ();
I have tried for a couple of weeks to resolve the error at runtime, but I have not succeeded.
Please, could someone give me any suggestions?
In order to replicate the error:
- Create a new Project in PyCharm
- Install PyTorch: pip3 install torch torchvision torchaudio
-Write the following code:
import torch
import torchvision
# An instance of your model.
model = torchvision.models.resnet18()
# An example input you would normally provide to your model's forward() method.
example = torch.rand(1, 3, 224, 224)
# Use torch.jit.trace to generate a torch.jit.ScriptModule via tracing.
traced_script_module = torch.jit.trace(model, example)
traced_script_module.save("D:\\resnet18.pt")
-Download Libtorch: Stable (1.10), Windows, LibTorch, C++/Java and CPU (release mode).
https://download.pytorch.org/libtorch/cpu/libtorch-win-shared-with-deps-1.10.0%2Bcpu.zip
-Extract the zip in your PATH.
-Create a project in Unreal Engine 4 named MyProject of type ThirdPerson and of type C++ (the default project is in mode Development Editor).
-Close the editor of UE4
-Open the properties in Project → MyProject Properties. Select the configuration Development_Editor and VC++ Directories.
-
Add the paths in External Include Directories:
(PATH)\libtorch-win-shared-with-deps-1.10.0+cpu\libtorch\include
(PATH)\libtorch-win-shared-with-deps-1.10.0+cpu\libtorch\include\torch\csrc\api\include -
Add the library int Library Directories:
(PATH)\libtorch-win-shared-with-deps-1.10.0+cpu\libtorch\lib -
Add PublicAdditionalLibraries int the MyProject.Build.cs:
public class MyProject : ModuleRules
{
public MyProject(ReadOnlyTargetRules Target) : base(Target)
{
PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;
PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "InputCore", "HeadMountedDisplay" });
PublicAdditionalLibraries.Add("(PATH)\\libtorch-win-shared-with-deps-1.10.0+cpu\\libtorch\\lib\\torch_cpu.lib");
PublicAdditionalLibraries.Add("(PATH)\\libtorch-win-shared-with-deps-1.10.0+cpu\\libtorch\\lib\\c10.lib");
PublicAdditionalLibraries.Add("(PATH)\\libtorch-win-shared-with-deps-1.10.0+cpu\\libtorch\\lib\\torch.lib");
}
}
-
Paste the dynamic libraries (dlls) in the Win64 directory of the UE4 project: (PATH)\MyProject\Binaries\Win64
Librarys: asmjit.dll, c10.dll. fbgemm.dll, libiomp5md.dll, libiompstubs5md.dll, torch.dll, torch_cpu.dll and uv.dll -
Paste on the top of file MyProjectCharacter.h: #include <torch/script.h>. If you Build the project you get a lot of notes, warnings and errors.
-
Replace :#include <torch/script.h> with:
#pragma warning( push )
#pragma warning(disable: 4582)
#pragma warning(disable: 4583)
#pragma warning(disable: 4458)
#pragma warning(disable: 4624)
#pragma warning(disable: 4800)
#pragma warning(disable: 4541)
#include <torch/script.h> // One-stop header.
#pragma warning( pop )
- Add the follow code in the constructor of your MyProjectCharacter.cpp
torch::device(torch::kCPU);
torch::jit::script::Module module = torch::jit::load("D:\\resnet18.pt");
assert(module != nullptr);
std::cout << "ok\n";
-
Build the project, everything should run fine.
-
Add the following code after the code above:
// Create a vector of inputs.
std::vector<torch::jit::IValue> inputs;
inputs.push_back(torch::ones({1, 3, 224, 224}));
// Execute the model and turn its output into a tensor.
at::Tensor output = module.forward(inputs).toTensor();
std::cout << output.slice(/*dim=*/1, /*start=*/0, /*end=*/5) << '\n';
-
Select Debug Options and uncheck: Use the new Exception Helper
-
Build and Run, then, you get an error in runt time in: at::Tensor output = module.forward(inputs).toTensor();
-
If you continue you get a new runtime error:
Unhandled exception at 0x00007FFE8618F199 (ntdll.dll) in UE4Editor.exe: 0xC0000374: A heap has been corrupted (parameters: 0x00007FFE861F77F0).
-If you continue your project your project will run normally. (for example: Question Answering: BERT(Bidirectional Encoder Representation with Transformers) with LibTorch & UE4 - YouTube)
- I have tried for a couple of weeks to resolve the error at runtime (for example, disabling optimization with #pragma optimize ("", off)), but I have not succeeded. The error in the heap continues.