OpenCV in Unreal Engine 5

Development Programming & Scripting
1

Hi,
I have been using OpenCV within Unreal Engine C++ scripts following this integration method. Today I downloaded UE5 and sadly it is not working.

I used following C++ actor for testing:
TestActor.h

#pragma once

#include "CoreMinimal.h"
#include "GameFramework/Actor.h"
#include "opencv2/core.hpp"
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include "Log.h"
#include "Misc/Paths.h"
#include "TestActor.generated.h"

UCLASS()
class ATestActor : public AActor
{
	GENERATED_BODY()

public:
	void TestOpenCV();

public:
	ATestActor();
	virtual void BeginPlay() override;

};

TestActor.cpp

#include "TestActor.h"

ATestActor::ATestActor(){

	//TestOpenCV();

}

void ATestActor::TestOpenCV(){
	FString RelativePath = FPaths::ProjectDir();
	FString FullPath = IFileManager::Get().ConvertToAbsolutePathForExternalAppForRead(*RelativePath);
	std::string path(TCHAR_TO_UTF8(*FullPath));
	UE_LOG(LogEndoVRCore, Log, TEXT("Testing OpenCV..."));
	cv::Mat img = cv::imread(path+"ThirdParty/Data/Lenna.png", cv::IMREAD_COLOR);
	cv::imshow("Display window", img);
	cv::waitKey(0); // Wait for a keystroke in the window
}

void ATestActor::BeginPlay(){
	Super::BeginPlay();
	TestOpenCV();
}

UE5 is running UE_LOG(LogEndoVRCore, Log, TEXT(“Testing OpenCV…”)); but then it is stuck.

2

There is already on OpenCV plugin distributed with the engine (plus one linked to the Composer). Did you check it out? Regarding your integration, I would start with placing a breakpoint after the UE_LOG and then stepping in this see what happens. Is path properly formed? Does it return a valid img but gets stuck afterwards?

4

Hi, Can you share this any link regarding this opencv plugin ?

5

Late response here

Yes, on your Build.cs you need to include: OpenCV, OpenCVHelper, then to include the headers you need to use this:

#if WITH_OPENCV
#include "PreOpenCVHeaders.h"

#include "opencv2/calib3d.hpp"
#include "opencv2/imgproc.hpp"

#include "PostOpenCVHeaders.h"
#endif

Finally make sure the OpenCV plugin is enabled, either on your project or add the dependency on the .uplugin file

5 Likes
6

Don’t think is late, thanks :wink:

1 Like
7

My guy, where is documentation on ANY of this?

1 Like
8

That is a funny one, historically UE does not have real documentation since UE1, the documentation is reading the source code. on visual studio try to look for key words and start reading, I strongly suggest to use raider, since its search engine is far powerful than visual studio

3 Likes
9

Yeah, I guess the question was rhetorical. Was just hoping for a miracle. Chat GPT probably knows.

1 Like
10

Hi sorry for bringing this up again.

“on your Build.cs you need to include: OpenCV, OpenCVHelper”
like the regular syntax correct ?
If you have a screenshot it would be great

If we are using the tutorial by jordan duncan , it uses an older version of UE. There is no inbuilt UE openCV plugin at that time. So do we still need to enable the plugin from the plugin editor ?

I’m trying Jordans method for UE 5.3 so im not enabling the plugin in the plugin settings.

Thanks for any help.

b

11

Ok so if anyone is still reading , here are my getting started questions.

  • As of 2024 should we still be using external opencv library (410 as of today) or the built in one in UE 5.3/ 5.4 what is the diffrence in the two approaches

  • As per this blog Use OpenCV in Unreal Engine we only need to enable the built-in plugin in 5.3 , add the dependencies in the build.cs file and add the code. His code works but it doesn’t call cv::imshow(); or use the webcam so I cant be sure

  • When I test @iFzic code above I’m getting a crash on cv::imshow(“Display window”, img);

  • If I wanted to use the latest version of opencv (410) would I still need to enable the plugin from the editor
    or use the method shown in this old video by jordan duncan UE4 and OpenCV Part 1 (Integration)

Thank you for any clarifications.
b