Dear all,
Having heard promising stories about CR I set out to give the Trial Version a go as of yesterday. Unfortunately, however, I have ran into the same error message twice. It appears at the very end of the Reconstruction Step (90%+ finished) and states something in the order of ‘Out of Memory’. I have run the same project last night and when I got back at the office this morning I found the program had crashed again following the exact same error message. I find the error rather strange though as the system we are using is less than a month old and provides well above average specs:
CPU: i7 6850K (15mb cache, 3,6Ghz, 6 cores)
GPU: 2x Nvidia GeForce GTX 1080
RAM: 64GB 2400MHz (4x 16GB)
HDD: 3TB (this is where the data and project files are stored)
SSD: 250GB (only running the OS and CR)
The project I am processing consists of 500 images taken with a P4P (20MP), forward and sideward overlap is 80%. The scene (which is only 100mx100m or so) was captured flying an across grid with the camera pointed at a 70 degree angle. Furthermore two orbits were flown around the scene at a lower altitude and camera angle.
Does anyone have an idea what might be going on here and, more importantly, how I can get rid of the error? It’s rather frustrating to see the program crash twice after running for nearly 9 hours!
Check where RC puts its cache. Its likely on the SSD and thus fills it entirely causing the error.
I have come up against this quite a bit. though i sort of have a solution now.
well 2 options anyway.
1st option, increase virtual memory (having more ram is better option). this fixes it sometimes, but on bigger scenes it won’t help. 100m x 100m isn’t that large. so maybe this will work, but keep in mind cache and virtual memory really needs be on ssd’s. and you are probably pushing it for space to do both. I’ve had virtual memory use over 700gb in the past. if this works, it will still take a long time, and much longer than it should. there is some bug there.
2nd option, create smaller region renders, I’d suggest splitting it into a 3x3 grid. there isn’t really an easy automated way to do this. but it can be scripted by editing region files.
2nd options works much better, and scenes can go much bigger without going over ram. but you have to deal with seems. which could be a problem.
good news is, that you shouldn’t need to re-do depth maps, as long as you haven’t cleared cache. so it should be much faster to go again.
there is a problem on large scenes, when
ShadowTail wrote:
Check where RC puts its cache. Its likely on the SSD and thus fills it entirely causing the error.
This may sound like a really stupid question, but how can I check where it locates the cash and/ or how I can I change it?
Hi Bob van der Meij
You can check the CACHE location by inspecting the WORKFLOW->SETTINGS->CACHE LOCATION
Wishgranter wrote:
Hi Bob van der Meij
You can check the CACHE location by inspecting the WORKFLOW->SETTINGS->CACHE LOCATION
Found it! It was set to System Temp by default. I can chose to select either Project Folder or Custom. I reckon the Project folder should do as this is located on my HDD with over 2TB of free space?
Yes.
Pick the disk where you have the most free space as the cache can get rather large during processing.
ShadowTail wrote:
Yes.
Pick the disk where you have the most free space as the cache can get rather large during processing.
Got it, it’s up and running again. Fortunately it started halfway (after depth map reconstruction), which should save me 2 hours. Does changing the cache location affect processing time in anyway?
it will if you have moved it from ssd to hard drive.
it will process a lot faster if cache is on ssd.
but that’s only useful if you don’t run out of space.
Has the issue been resolved by moving the cache?
Götz Echtenacher wrote:
Has the issue been resolved by moving the cache?
Not sure yet. It has been processing for slightly over 3 hours now. I reckon approximately another 3-5 hours remaining. The error used to present itself at the very end (>90%) completed. I guess time will tell if it will succeed this time ;).
Oops, didn’t realise it’s so current…
Good luck then!
Too bad, it crashed again at 82%!
The error message reads as follows C:(…)/RealityCaptureCrash_6384_45833-666_1_0_2_2600_Demo RC.dmp
Any ideas? I have already uploaded my dumpfile to CR but have not received a reply yet.
Well, with your setup that should not be a problem at all.
So I don’t think it’s the hardware per se.
Are you sure your RAM is ok?
Wishgranter should be able to help.
In the meantime, did you look at those yet?
Avoiding “Application ran out of memory”
BUG 501 - crash on calculating model at “high quality” (Crash at 81%)
go to your info panel, and adjust the region size.
then split it up into a few different parts. it will reuse the cache each part shouldn’t take too long.
you will need to think about how to deal with the seams though.
Hey chris,
the links I provided suggest to reduce the size of each part of the mesh.
I think essentially it is the same effect, only that RC merges them automatically.
But I really can’t imagine that it should be down to his hardware - I am much more humbly equipped and never had any issues.
At least with this error! hehehe
its possible there is part of the mesh that is giving issues.
probably out on the edge somewhere.
I’ve had times splitting up the into lots of regions, one of them still crashed.
could be miss aligned or not enough information. and it gets stuck.
just making it 80% of the original size may fix it too. but you get keep doing new regions next to each other.
Huh, that’s an interesting idea.
Although I did find it strikting that the percentages are almost identical.
That could also be a processing step…