Hi, I was thinking of using Unreal Engine for some computer vision/image processing related simulations I am currently working on, given how realistic the environments look etc and the amount of control the user has on changing the scenarios. For doing this, I will be needing multiple viewpoints of the environment (similar to having multiple cameras in real life looking at a scene), and I would like to either log images at around 30 Hz, or a video (if it’s simpler) from all the cameras as they move around in the scene. Can anyone let me know if this is possible/easy and hopefully point me to some resources about how to get started with this? Thank you!
Hello saihv,
I believe I understand what you are trying to achieve as far as what you want done within UE4. However, if you would please clarify as to the end goal of why you are wanting these vision/image processing simulations. What do you mean by " logging images? "
It is possible to set up multiple cameras at various positions and angles depending on what you want. However, changing the image quality of your photos within UE4 is impossible. We typically see a workflow of images that have had all alterations done before they are imported to UE4.
If you went with creating a video you are subject to AVI’s or image sequences that can be taken and then manipulated in another software. After you export them from UE4.
Yes, for what you are trying to achieve it is possible. However, it is extremely difficult and would require knowledge of a variety of different facets that out software includes. To name a few Lighting, Post Processing, Matinee, Materials, Cameras and how they work within the software, Rendering, image manipulation working within the restraints of UE4. If you are wanting to manipulate images or video, especially if you have a deadline, I would start researching into the rendering side of using our software. From there you can formulate an opinion of what you would need for your specific project and then make a decision on whether to use UE4 or not.
Here is a link to our wiki which will lead you to other users issues and documentation for any questions you may have that pertain to specific parameters and features that you will need when creating your project.
Hi ,
Thanks for the reply. I feel like I’m missing something here, because I did not expect it would be that complex. Here’s a little more detail on what I am trying to do.
Assume there’s a simulated world of objects. I need two or more cameras separated by a certain distance from each other, which look at this “world” and continually save images that they see, which is what I meant by logging. Based on how the images of the same scene look from camera 1 as compared to camera 2, I would be doing feature extraction, matching and some other computer vision techniques, so that in the end, I can get accurate information of all the cameras’ exact positions in the world, even as they move about randomly. The real life application of this is: assume there are multiple aerial robots that have cameras on them: this “position estimation” technique will help all the robots understand where each of them is even in unknown environments. Also, grabbing these images would be easier to do in a simulated world than in real life.
I don’t need to change any image quality related things, honestly, all I need is multiple viewpoints that can be controlled in terms of their positions in the world. As I have only worked with robotics simulators in the past, my impression of a simulator usually involves me “placing” or creating some simulated objects, like houses, let’s say, and then viewing that scene through different viewpoints. Hope this makes sense, and hope I am not confusing myself with a completely wrong impression of what Unreal can do for my application. 
Thanks!
It is theoretically possible to do what you want with UE4. The engine is certainly capable of capturing video and then passing that along in a number of ways. Most noticeably is Matinee. Matinee is our video movie making platform within UE4. This feature is capable to capturing information and then exporting it from Unreal along with many other functions. Another way to pass along information is through use of a scene capture. This can then be passed to a render target and then displayed in a material.
However, the limitation comes in when it comes to the actual storage of the information captured. UE4 is not equipped with a functionality that will capture, store, and then pass along information into an array. From there you are wanting to allow information to be accessed by different actors within your project. This way you can access this array and allow, through some coding/visual scripting, for different actors to then tell where other actors in the world are.
UE4 does not have the means to facilitate what you are after without a serious amount of change to the base code. As I stated in my last comment you would be putting much more work into this than perhaps you were intending by getting the source code for UE4, altering it without breaking the program, making sure it all worked and then actually getting to your project itself.