Using Unreal Engine 5.0.3 or 5.1.0 built-in OpenCV plugin

Development Programming & Scripting C++
, , ,
1

I’m trying to use the built-in OpenCV plugin for Unreal Engine 5.0.3 or 5.1.0.

I added the plugin to the project level build file, like so:
PublicDependencyModuleNames.AddRange(new string[] {
“Core”,
“CoreUObject”,
“Engine”,
“InputCore”,
“RHI”,
“RenderCore”,
“Media”,
“MediaAssets”,
“OpenCV”,
“OpenCVHelper”
});

I am able to include the following import statements and compile:
#include “opencv2/core.hpp”
#include “opencv2/highgui.hpp”

However when I try to launch the editor, it loads up to 75% Initializing and then crashes with this error:

Unhandled Exception: 0xc06d007e

KERNELBASE
UnrealEditor_QRCode!__delayLoadHelper2() D:\a_work\1\s\src\vctools\delayimp\delayhlp.cpp:312]
UnrealEditor_QRCode!_tailMerge_opencv_world455_dll()
UnrealEditor_QRCode!ACameraReader::ACameraReader() [C:\Users\matth\Documents\Unreal Projects\QRCode\Source\QRCode\Private\CameraReader.cpp:8]
UnrealEditor_CoreUObject

I’m not sure how the D drive path is related.

If I comment out: “cv::Mat cvMat;”, the project will launch, but of course I need access to the cv::Mat type.

“cv::Size cvSize;” works though.

If anyone has been able to get OpenCV working in Unreal Engine 5, I would love any insight and help.

2

Hi. Im also currently doing something using OpenCV in Unreal.
I got my project to compile by also including the Modules as a private Dependency.
And then I found in a video that the includes for OpenCV should apperently look like this:

#include “PreOpenCVHeaders.h”
#include “OpenCVHelper.h”
#include <ThirdParty/OpenCV/include/opencv2/imgproc.hpp>
#include <ThirdParty/OpenCV/include/opencv2/highgui/highgui.hpp>
#include <ThirdParty/OpenCV/include/opencv2/core.hpp>
#include “PostOpenCVHeaders.h”

Link to Video

You do not need all of the includes, but I still but them in so that people are able to detect the scheme.

I did not find any documentation on using OpenCV with UE. So if you know some good links these would be apprecieated. Currently I am figuring everthing out myself. Wich is more time consuming than I anticipated.

3 Likes
3

thank you for your comment. does it work with all the include. and can you use Mat after that?

4

I have the same issue using UE5.2 built-in OpenCV, and solved it by copy the opencv_world455.dll from UE_5.2\Engine\Plugins\Runtime\OpenCV\Binaries\ThirdParty\Win64 to my project’s Binaries\Win64 Folder.

3 Likes
5

TSM. This post is appreciated.
Why after so many years that CV has been going around there are no dedicated features/plugins to make this easier?
I’ll follow this thread as I am also looking to avoid UI controllers.

6

I have have a OpenCV Windows plugin to read QR code.

But you need to change these in order to work.

"Engine\Plugins\Runtime\OpenCV\Source\ThirdParty\OpenCV\include\opencv2\core\utility.hpp"

* 52 = #if defined(check)
* 53 = # warning Detected Apple 'check' macro definition, it can cause build conflicts. Please, include this header before any Apple headers.
* 54 = #endif
* 934 = bool check() const;

I am working on Android support as well but I can’t give any promise about it. Because there was another clash, too

1 Like
7

I meant, why isn’t this implemented in Unreal?
However, I’ll take a look at your code.

1 Like
8

I set it private because I used a marketplace plugin’s android build system for ZXing side of my QR processor.

But you can check my opencv implementation for QR decode.

#ifdef _WIN64
bool UFF_QR_ProcessorBPLibrary::OpenCV_QR_Decoder_Callback(FString& DecodedString, uint8* Buffer, FVector2D ImageSize)
{
    if (!Buffer)
    {
        return false;
    }

    if (ImageSize.X == 0 || ImageSize.Y == 0)
    {
        return false;
    }

    const cv::Mat Image(cv::Size((int32)ImageSize.X, (int32)ImageSize.Y), CV_8UC4, Buffer);
    cv::Mat Points;
    cv::Mat Output;

    const cv::QRCodeDetector QR_Detector = cv::QRCodeDetector::QRCodeDetector();
    DecodedString = QR_Detector.detectAndDecode(Image, Points, Output).c_str();

    return true;
}
#endif

bool UFF_QR_ProcessorBPLibrary::OpenCV_QR_Decoder(FString& DecodedString, TArray<uint8> Buffer, FVector2D ImageSize)
{
#ifdef _WIN64
    if (Buffer.IsEmpty())
    {
        return false;
    }

    return UFF_QR_ProcessorBPLibrary::OpenCV_QR_Decoder_Callback(DecodedString, Buffer.GetData(), ImageSize);
#else
    return false;
#endif
}

Actual decoding is in callback because I developed an Intel RealSense plugin for UE5 and I can use this with it to decode QRs. I use callback one in realsense’s FRunnableThread and feed with uint8* buffer from sensor without copying data to TArray.

1 Like
9

This actually solves my similar problem.