High detail reconstruction issue

Development Asset Creation
,
1

I know this has been brought up before but I have not been happy with any response to how it relates to my problem.

I’ve noticed with high detail reconstruction after 2 hours or more of high CPU gpu intense calculations RC hardly works my computer at all.

CPU usage goes to zero most of the time, GPU related calculations are completed I believe the process is around 25% complete . Now the next day or two RC just rolls along hardly using my PC to complete the process.

Seems very wasteful on resources so just wondering what’s going on…
1600 Raw images being processed with a i7 6700k , 2 980tis and 16GB DDR4 ram.

Thanks !

2

It pushed another day now, its really my ram usage goes from 30%-80% during the day and my CPU almost never gets used or GPU. sometimes i see my harddrive working at around 5-30MB/s. 

If my ram is hardly being used why on earth is this taking so long? it seems like there is a bug?

 
I should note the reconstruction is of a large interior floor.

Should I assume just pausing and resuming should solve the problem as per the FAQ for large scenes? is this being addressed in a future release?

Thx!

3

so are you saying you are getting this issue 25% into model creation?

by that time all the gpu part is finished so you shouldn’t see it in use. (only used for depth maps).

are you running hard drive or ssd for temp?

it could be a ram issue. it probably wants a fair bit more with 1600 photos.

I think the last part of model creation is single threaded. but that shouldn’t be seen until 90%.

I’ve seen this plenty of times. just not at 25%, usually its 80-90% for me. but I’ve only been using it with either 64 or 128gb ram.

also check your alignment. if it is working on a part of the mesh thats not alignment properly it can cause these kinds of issues. can be also be solved by not reconnecting the problem area. though finding that bad part can be difficult.  

4

the temp files should be stored on my SSD the project files on a normal hard drive. 

I think the alignment should be good? The control points all match up and the camera details look good.

I would say it gets to 30% then just takes absolutely forever to do anything with hardly any CPU/Ram/GPU utilization. 

For example reconstruction on normal for 600 photos takes 1-2 hours but high took me 1 full day. This project has 1600 photos and its slowing down the exact same way but now takings days.

I think im going to try the pause restart method first, see if that sparks the program to go again at full speed. I mean im within the requirements for the number of images 16GB is fine for 1600 photos as per the FAQ.  (I may have forgot to set the reconstruction zone though :S)

How do I check the alignment? the initial alignment and points looks pretty good. (Though I did noticed there were 10 images stacked in one spot that seemed odd)

Thx

5

select all the cameras and use the inspect tool.

you should see lots of lines between lots of cameras.

it may show some areas with less connections that could be an issue.

1 day dosen’t sound too bad for high detail.

what res are your photos?

i find it normally 1 - 3 days high reconstruction for 2500 with 8 core, 128gb ram. 

if using cli, and going 5000-10000 photos can spend 3 weeks on it.

so 1 day with only 16gb of ram dosen’t sound too bad.

but have a look at your commit size. (total ram+ virtual memory). that should tell you how much ram it would ideally like. if its having issues somewhere, it can need a lot more ram to get through those bits without crashing.

but still i think i have less issues with more ram. or i get the issues later. but I still get them even with 128gb. so adding more ram won’t necessary fix it. I think rc could do a lot more to deal with this issues better. its probably my biggest issue with rc. its wasted a lot of my time.  

I remember a long time ago there was talk of having the mesh save as it got created. so you had everything up until it crashed. it would really help to figure out where the problems are. and also give you something usable even if much smaller area and just not waste so much time.

never really made it in to any updates though.

 

6

Photos are taken from a Canon 60D in Raw (18MP). DDR4 is through the roof right now =/ but ill have a look later, i dont notice it ever pushing 16GB but my page size is always 2-4GB in windows for some odd reason.

I just hate sitting around waiting haha =/ especially when i notice my PC idling most of the time from 30%-100% (This never happens in normal)

7

you may just need to leave it for a few days to finish.

 

 

8

I have a same problem like Mike Simone describes. 

PC1: Ryzen 1600X, GTX1060 6gb, 64Gb DDR4, Cache HDD
PC2: Xeon 2630v3 x2 (socket 2011-3), GTX1080Ti 11gb, GTX1070 8Gb, 64Gb DDR4, Cache SSD
PC3: Xeon 2620 x2 (socket 2011), GTX1060 6gb x2, 147Gb DDR3, Cache HDD

PC1 runs RC project with:

  • 1615 photos 42mpix JPG
  • 2 scan positions PTX

Now it’s on Calculating Depth Maps (CDM) stage and it runs more than 50 hours at this moment! For last 24 hours CDM process pass from 68% to 72% And GPU usage for this 24 hours is not more 35%. You can see usage of GPU (look to Elapsed time):
PC1 (reminder): Ryzen 1600X, GTX1060 6gb, 64Gb DDR4, Cache HDD 

 

PC2 runs RC project with: 

  • 1434 photos 42mpix JPG
  • 4 scan positions PTX

CDM stage, for last 24 hours passed from 85% to 95%, and most of time GPU usage is near zero
PC2 (reminder): Xeon 2630v3 x2 (socket 2011-3), GTX1080Ti 11gb, GTX1070 8Gb, 64Gb DDR4, Cache SSD

PC3  runs RC project with: 

  • 2400 photos 42mpix JPG
  • 19 scan positions PTX

CDM stage, for last 24 hours passed from 85% to 95%, and most of time GPU usage is near zero
PC3 (reminder): Xeon 2620 x2 (socket 2011), GTX1060 6gb x2, 147Gb DDR3, Cache HDD

What it’s all about? I can tell - something wrong here. CPU, GPU, RAM usage most of time is not optimal, below 20%. When Calculating Depth Maps just start it uses all my GPUs for 100%, but after some time it almost stops. What happening then I cant see, RC slightly use my recources and seems to be stucks. 

  • All reconstructions in High Detail
  • Few times restarted the project
  • HDD\SSD1: Windows 7
  • HDD\SSD2: Virtual memory 65 536 Mb - 524 288 Mb (testing this configuration few days)
  • HDD\SSD3: RC Cache HDD\SSD

 

My personal spreadsheet with approximate values:

|

Processing stage

|

CPU use %

|

RAM use %

|

GPU use %

|

HDD use %

|

% of time

|
|

Aligning Images

|

20-100

|

20-100

|

0

|

20

|

5

|
|

Calculating Depth Maps

|

100

|

100

|

100

|

100

|

30

|
|

Grouping Depth Maps

|

100

|

10

|

0

|

10

|

15

|
|

Decomposing for Large Scale Processing

|   |   |   |   |   |
|

Creating Model

|

100

|

100

|

0

|

10

|

40

|
|

Texturing

|

80

|

80

|

mem100%

|

100

|

10

|