Regarding cameras and data output from them.

I have not found the support section of the forum so I’m guessing the general discussion is it ?
I am very new to UE and I just want to know some things to know if it’s possible or not in UE.

Okay so I have a bunch of cameras and I want to grab data from them ? and send data to NP arrays ?
So these cameras just lay there in the background sort of untouched and collect data and then send data to Arrays.
Sort of security cams work but the data they send is to these Numpy Arrays.
I want each character to have a camera on their head and to send live data with the cam to these arrays, and then process these arrays and shape actions of character.

From what I understand I can use python now with UE ? and Numpy ?

So my idea is, send data from cams to a 2d array in real time, is it possible in UE ?

Glad to hear about this.

Glad you like it but I really need an answer if it can be done, so I can get with the basics of UE, start off.

Hi @GigiK-K-K

Did you take a look into **TextureRenderTarget2D **?

Yes I did, I tried to make a blue print with it but failed**,** there are no errors but it does not grab the capture device. I probaly don’t know how to do it and there is no real documentation on it.

I tried to search for blue prints regarding camera feed to array but did not find out anything. I can make an array and fill it with data but not from a feed source, there is an option, I see this can be atuo created like with: “RenderTarget2Darray” that is a component in the blueprint, I don’t know how to hook up the cam and then make it “print” out the integer numbers in output log. I don’t even know if the array stores data from the cam.

So I made a security cam that has a camera and a screen monitor where the image of the live cam projects on the monitor/screen/
How do I go about this, I try from the camera directly with capture2d device to make a blue print and connect a RenderTarget2Darray but does not work.
Or do I try to grab the dynamic texture on the monitor and send it to an array ?

Can I connect the cam to an array directly and output in text on screen live or inside the log, I want it to output the numbers in the form of integer (array numbers) representing pixels, the rgb values. Or do I have to get it from the texture that is changing live on the projected screen/monitor, and print out the numbers from that, do I create a blue print for texture ?

What I’m trying to do:
I’m trying to get feed data inside the 2darray that is type integer (numbers) either from cam directly or from the material/texture where cam projects and then print out the numbers in the output log from the array, that is all. This for me to see if I can do this in UE4. Later I will use this to shape character behaviour in games, but I will have to dig deeper into that, based on the world they see around them. For now I want to know if I can get camera pixel data inside the array and see if it’s inside by printing it out. So I’m trying to access pixel data from camera feed or maybe from texture and load it into array and print out the values of the array, so I can verify data inside array

Regarding what I did (wrong probably for sure)
I uploaded photo 1,2 with blue print and scene.
Photo 1 shows camera attached to a ball as a capture2d device and on the left the screen with what camera has in front of it capturing, cubes, cilinders, spheres, it’s on the screen, that is **TextureRenderTarget2D.

The blue print of the SceneCaptureComponent2D shows the array I made and that is type integer (digit numbers) and shows the blue print with the capture device first connected then to the create redner target 2d that can create an array then I loop the array and try to print it out in the output log as numbers from the x/y row column 2d array. I don’t even know if it stored data since I can’t print, or if the rest of the setup is bad regarding the print function.**

I’m guessing my blue print is not good, because I don’t know how to do it since I’m a beginer with these blueprints and python is limited inside UE4.
Also there are really no examples at all for unreal regarding “getting camera feed, storing it as pixel integer values in a 2d array (pixel data rgb) and then printing the numbers out.” All I could find is 2d Array that makes a number of objects, or cubes.

Maybe I don’t need rendertotarget2d and this can be done some how else ?

C++ is not my thing (not that is not my thing but I never worked with it), so I thought maybe doing it with blueprint is “easier said than done” since it handles the C++ code.
I have to get with C++ and learn extra now, it’s fine as long as I know “This works in UE4” to get data from a sitting cam, any cam, get the pixel data, insert data into array and then do data manipulation there, work with arrays.

After did some research it seems you can do that in c++



TArray<FColor> SurfData;
auto RenderTarget = RenderTexture->GameThread_GetRenderTargetResource();
RenderTarget->ReadPixels(SurfData);


It’s look like what you wanna do right ?

Okay, what I want so I can explain better from different points of view.

I want to get anything into that 2darray for now as practice, but mainly the camera pixels (rgb), if possible directly from the cam, if I can’t get that directly inside the array then the texture render 2d target where the camera renders the image on the texture.

This data go’s inside as an image of pixels, the array stores the pixels in columns and rows (x and y) as numerical values, this is a 2D array.

I’m new to unreal, this blueprint stuff is confusing to say the least with arrays because I can’t seem to find a way to even get a texture inside a 2d array, store it then print it’s content out inside the output log. So I was trying to store a regular static texture as practice inside a 2d array and I could not do it with blueprint, I looked over the internet tutorials and I could not find how one could store an image, a texture or direct camera feed insdie a 2D array with blueprints. There are no tutorials on that regarding blueprints.

To store an image into a 2d array as I know it requires to extract the pixels first and create a buffer that you feed to the 2D array. Then the pixels can be inserted row by row inside the 2D array.
I can’t believe there are no tutorials on this.

I will take a look at your code after I setup visual studio, the Tarray means Texture arrary I’m guessing, I’m not familiar with it, and I have not worked with C++ ever before, so I thought blueprints is the way to go for now, I know of NP arrays (numpy arrays) but these are not inside Unreal ?
With numpy it was very easy, just feed the buffer to array and it’s done, all get’s translated by numpy inside and array gets populated.

Something like arr = np.array(some_image.jpg, pixels :]) and the image got stored in array as numerical pixel data with rbg values.
I’m guessing I can’t do that in unreal, it has python but without it’s numpy counterpart and python here as I understand only deals with some functions.

Tell me the code you put up, is it the dynamic texture from the render output of the camera ? The texture that get’s imprinted after the camera renders to it ?
This is a dynamic texture, so we can store it live inside a 2array. This is sort of what I want, and as the camera view changes so does the dynamic texture that the camera projects onto and if the texture gets sent to the array live so does the array change meaning we have changing digits inside the array live as the camera moves around, we could hook up and interogate those x\y coords live and do data manipualtion

This is what I want, but for now just get the data inside there live inside the array.

Maybe the Tarray does not store pixels but some other form ? just asking, there are different 2D arrays, a 2D arrays is 2 dimensions it has x\y plot coords as opposed to 1D array that is complete linear array as a list of things. This Tarray, can it store the rbg pixel values ? Maybe it stores data in other form like lists of vertexes.

C++ 2d arrays are different than np 2d arrays or any 2d array in py.

Rows and columns are different
The columns in np array are rows and you count where you want on the row, and columns go vertical and represent each row where as in c++ is different the columns are actually the position on the rows.

Each row should be a column as you go down vertical, not in C++ the columns are actually the numbers on the rows.

For example I have a row, and each row is a column. This example has 3 rows and each row represents a column.

Column 1 = 1,2 3,4
Column 2 = 5,6,7,8,
Column 3 = 9,10,11,12

this has 3 columns made of rows, and each row is 4 digits long.

if I want to access number 7 for example x/y I say column 2 row position 3, it actually starts from 0 but this is a good example.
x/y plot 2,3

number of column and position on the row, where as in C++ is different the column is actually the row number, why would they do it like this since column means vertical, going vertical on the index.

I just want it to do it with blue print, grab the target render texture and send it inside of array, but C++ is an array inside an array, works different…

So me confused :slight_smile:

I find blue prints anoying anyway, so I think I’m off to C++ learning tutorials.
Well anyway 2d Arrays in C++ are totally dif…than py ones.
https://youtube.com/watch?v=B3iC40frU4MHe says columns are the position number on the row. But I think he explains wrong maybe, and 2 represents the number of columns and 3 how long the rows are. Anyway, this does not help me but it does sort of, to insert data into an array I must know how a 2d C++ array works.

I don’t like blueprints, I could not find any way to grab the texture that imprints on the plane/wall and send it to a 2d array.
You have all these components that overlap with all those wires, Crazy :slight_smile: and just like in UPBGE the documentation on these things is really brief and robotic on the Unreal documentation site.

If you could help me or anyone make a blue print with capture2d device to output inside a blue print. I saw that it may be possible, to cast the 2d device to a render-target inside the blueprint then that render target to 2array, all inside blueprint, I see the componets there but I have no idea how.

I just put the components there to show they are all there and it could be done ? but I don’t know how to connect them inbetween because they need other stuff as well. So I don’t need to make a render target2d outside visible, as I understand it could be done just with blueprints, with a rendertarget inside blue print from the camera blueprint where inside the blueprint there is the 2d render target that is sort of just the value without actually the texture on the material plane to project to but just internaly where it gets casted inside the blueprint then from there to 2d array ?

If you could look at the picture I uploaded, all the components are there but I have not connected anything, just put them there to say what I think of them sort of and if it could be done ? Again between these components there are probaly other components like loop and other stuff.

Now this I have installed visual studio.

Where do I paste this, I have no idea how it works I am new to C++ and unreal.



TArray<FColor> SurfData;
auto RenderTarget = RenderTexture->GameThread_GetRenderTargetResource();
RenderTarget->ReadPixels(SurfData);


Could you guide me a bit with this code.

Also down here the map with the components I made that I told you, I marked stuff with the boxes.
If you look at part A I can select the actual texture where (Set) texture name is “coasas” this is the render target texture used by me to project on the wall inside the scene, so camera projects on this as\like security cam, so it let’s me select it, or maybe I don’t need it and I can make one in blue print without actually creating one on the scene ? Well anyway I can select that and get it to array ? somehow by using rendertarget2d array ?

Well anyway, if I can select it then there must be a way in blueprint to convert it to integer to place it inside 2d array ?

Here is the actual texture.
It is the texture that is used to be set on the wall/plane with the material, the camera or capture 2d device as it’s called as a componet projects on this texture, like a security cam would output on the monitor. It’s this texture I’m trying to send to a 2d array and get every changing instance inside the 2d array, meaning the array is constantly changing, the numbers inside of it fluctuate as the projection on the texture changes from the camera (capture 2d device)

Can this be done in unreal ?