I’m currently trying to use OpenCV inside a project and having issues around creating a cv::Mat
#if WITH_OPENCV
// IWYU pragma: begin_keep
#include "PreOpenCVHeaders.h"
#include "opencv2/unreal.hpp"
#include "opencv2/opencv.hpp"
#include "opencv2/core.hpp"
#include "opencv2/ml.hpp"
#include "opencv2/imgproc.hpp"
#include "PostOpenCVHeaders.h"
// IWYU pragma: end_keep
#endif
// Create a destination matrix for the blurred image
cv::Mat dst;
// Convert the pixel buffer to a cv::Mat
cv::Mat src = cv::Mat(cv::Size(TextureDimensionY, TextureDimensionX), CV_8UC4, PixelData.data());
initially i thought the PixelData was incorrect but it seems that I can even create an empty Mat.
I am not familiar with OpenCV but feel I feel like I am close and has to be the way unreal handles memory allocations and possibly causing conflicts with OpenCV as the error is around disassembly code.
I found the problem was I hadn’t added the dll via being a module and as the DLLs and Lib files only get added to the project if the plugin is enabled, once I fixed this it worked hope this helps someone else
Thank you for posting how you fixed the issue.
I am beginning my opencv journey and it has been an exercise in frustration ,so this might help me down the road.
Just so im getting this correct, Your issue was resolved because you enabled the opencv plugin that ships with ue5 correct ?
Did you also add a Thirdparty folder in your project.
I’m stuck on making ue5 opencv webcam work because I want to use a recent version of opencv.
I have downloaded all the files ,put them in the third party folder , created the build.cs file, but on testing the project is just refusing to open past 77%.
I take it your just building the project and not doing a debug build?
and for now i just went with using the unreal headers but manually made the dlls and libs in binaries from the unreal folder so i don’t need the plugin on at the moment while i transition the new libraries.
I delved into how unreal handles opencv and they have a few helpers to suppress warnings and so forth, but they don’t alter the opencv code like the tutorial.
if you delve in to UE code they have ifs to check if you are using open cv but as i am i removed them.
but the key was the pre and post opencv headers and the unreal hpp file.
this is the lib from my module being manually added, and i just copied the dlls into the binaries. I am currently working on having all the dll and etc in my module so i can upgrade things when i want etc. but loading dlls dynamically is not a straight forward process that i thought it would be
but i dont have opencv turned on anymore, but definetly look at the ue folder as some of the helper methods could be of interest to you and the opencv library in unreal is pretty complete on functionality, they just haven’t implemented 99% of it in the blueprint code and you have to code it.
hopefully it helps, sorry for the late reply and spellings just really slammed at work so i’ll do my best to keep an eye out for your posts but don’t expect instant reply
I take it your just building the project and not doing a debug build?
I’m launching the editor via visual studio. I.e build the project Ctrl +b
Then Debug > Start New instance .
Is this correct ?
Would you have a repo where I can see how you are using the webcam input ?
This is what I want to try :
Use case 1.
Open the webcam via opencv (or any other way), save the frame to a CV Mat object, Process the frame using a cv function (e.g edge detection ) , Save the frame as a unreal texture to do further processing
Use case 2.
Open the webcam via opencv (or any other way) , save the frame to a CV Mat object ,Process the frame using a cv function ONNX inference (Depth Estimation ), Do something in UE based on depth.
basically im stuck at getting the webcam to work and saving / processing the texture from the older tutorials out there.
Thanks once again for replying here and yes, i wont expect an answer right away (or at all.I’m glad you replied though ) .
so when you start in the debug instance, does it not load and takes you to a low level file filled with numbers and pointers? if so this would suggest you are missing the dll and libs in your binaries?
Have you looked at the OpenCV code in unreal yet does it have the needed functionality you are after?
and your code that calls upon opencv do you have the build.cs file pointing to the corresponding modules?
Unreal implements some helper headers to suppress compile errors associated with opencv like the check method and how it handles memory as it needs to conform to the unreal way so you dont get memory leaks
Hi,
When I start a new instance, UE splash screen opens up but stops at 75%. The vs c++ editor does not take me to a low level file filled with numbers.
I have the lib files in the binaries.
Based on your help via DM I’m going to try and un-follow some of the steps in the old video tutorial on UE & USB cam (commenting out the check function etc ) and use the opencv helpers instead.
Thanks, I’ll definitely take a look when I get the webcam working
) .
