I have a scene with 738 cameras. Many of them are large images – 4000*2000 resolution. They align well, but when I try to do a “Normal Detail” reconstruction, it chugs for a few hours and then I get “Processing failed: Application ran out of memory.” Extremely frustrating that I don’t know this is going to happen until the process has been running for hours, and there are no tips on how to deal with it. A few questions:
- Does this refer to system ram or GPU vram?
- What can I do to reduce the memory usage? Would changing the “detail decimation factor” help, and if so, do I want a bigger number or a smaller number? How about shrinking the Reconstruction Region a little? Do I need fewer points in my sparse point cloud (I have 1660879)?
I’m running this on an ancient machine: 2x Xeon E5430, 24GB DDR2, Quadro K600 (1GB vram) .
Ordered parts today to build a machine with GTX980Ti but I’d like to get this model done on the old machine if possible. I don’t need it to be high detail, in fact the sparse point cloud is almost good enough.
Hi Aaron Curtis
in short, to find the issue which causes this, we need a bit more info what issue created this behaviour.
set the UI like here: https://www.dropbox.com/s/dw7mnn0uagh8k … w.png?dl=0
then run the reconstruction again ( Depth maps are already calculated ) – this way it will reconstruct the model much faster.
and when you get the error warning, DO NOTHING except this !!
look for this file RealityCapture.log in the SYSTEM drive and send it to me, so that I can see where the error occured, so that we can take a look at it and solve it by just changing settings in RC.
Ok, it’s running now. I looked for RealityCapture.log from yesterday’s run but couldn’t find it. What’s the full path to it, usually?
Hi Aaron Curtis
I looked for RealityCapture.log from yesterday’s run but couldn’t find it. What’s the full path to it, usually?
It should be in the TEMP folder on C: drive. If you cannot find it, there is something wrong…
Just run it until it gets in the reconstruction issues and look for it on the system drive…
I tried with Detail decimation factor = 10 and it completed.
Then I tried setting Detail decimation factor back to 1, and had the out of memory problem again, but it crashed much sooner this time – after minutes instead of hours. I searched for RealityCapture.log but couldn’t find it. It’s not in my C:\TEMP, and C is definitely the drive with my windows installation. However, I took a screenshot and attached that in case it is helpful.
(By the way, my experience up to this point with RealityCapture has been really great!)
Hi Aaron Curtis
try to do a full PC scan for RealityCapture.log, but it should be located here: c:\Users\USER\AppData\Local\Temp\
so probably just windows search cannot find it… change the USER to you user account name used on that PC to get to this location.
As I can see there are plenty of images in the project. Can I inspect it if there are not have any issues with alignment ?
Make a backup save of this project and delete all components except the largest one and try to align the project again.
I have begun pushing my settings quite high and this has started to give me memory problems. I have only 16GB.
It seems like RC could do with some memory management settings to help users avoid this problem, I know this would come at a cost of processing speed but that is better than a complete failure.
Hi Stuart
if you get into reconstruction issues with 16 GB RAM, only then try to change RECONSTRUCTION->SETTINGS / the VERTICES COUNT PER PART from 5 million to say 1 million.
Hi,
I am having the same issue. I left my PC process a High Detail model overnight and now I have find RC telling me “The application ran out of memory”. I cannot see the drop box image above as it has been removed.
I will try and change the “VERTICES COUNT PER PART from 5 millions to say 1 million” but in the meantime, anyone have any ideas or recommendations to avoid this in the future?
I am using a 980Ti with 16Gb of RAM. Not sure if relevant, but I changed the Cache custom location from my C drive (SSD) to my HDD as I have terabytes of space on that HDD.
My project has 565 images (4000x3000 pixels), all aligned correctly forming one single component.
check your alignment.
do you have any bits that look misaligned in the mesh that you got to work?
this can seem to throw out memory requirements.
You may want to up your page file. if you have enough of this it won’t give this error. don’t let windows set it automatically. it will crash before windows will resize to a larger amount.
but its going to be much slower (use ssd at least), and it might go now where after a while if its a alignment issue.
Hello,
I have project with 7028 inputs (multiple components imported into single project). When I run alignment, every time I get the message “Application ran out of memory”. I have tried several alignment settings and there is no difference.
Error codes:
- Feature detection completed in 4666 seconds
Processing failed: The application ran out of memory
[Failed to detect features for image '\diskstation\scan-data\Pula\Pula_Arena_Plast_05042017\Alpha_arena\Vanjski pla [0x10003\0x10001]
[0x10003\0x10001]
- Feature detection completed in 79 seconds
Processing failed: The application ran out of memory
[0x13010\0x10001]
[0x13010\0x10001]
Reconstruction failed after 2310.446 seconds
Specs:
i7 4790
32gb ram
980ti
512gb ssd (project + cache) + diskstation (inputs location)
Do I need to increase pagefile (now is at 20gb)?
You may want to try and add another disk to the computer that has a storage capacity of at least 1TB and put the cache on that disk (settings!).
Such a large project will very likely generate more temporary/intermediate data than the small ssd can hold.
Hello Zlatan,
which Alignment settings have you tried? If you are trying to align them with Feature source set to Use all image features, there still could be the problem with RAM shortage. You can find these settings when you click on specific camera/cameras in the component.
You can find more information about these settings in application Help - Using components.
Yes, I use “Use all image features”, I have another pc with Xeon 1650 v3, 64gb ram, gtx 970, 1tb ssd. I will try there.
EDIT: It was fine with xeon configuration. I guess that lack of RAM was the problem
Zlatan
