CUDA error while texturing in new release

Development Asset Creation
,
1

I am getting a “no sufficient CUDA-capable device available” error about halfway through texturing some of my largest models (1000+images; 1000k tris).  This has only happened since I installed the new release.  I continue to be able to texture slightly smaller models without the problem. 

I am running with a GeForce RTX2070/431.70.

Adding to my confusion, I moved one of the models to an older computer with a GeForce GTX970M/382.05 and it completed without the error.

I’m hesitant to do any driver updates unless I am sure it is necessary since I am running a number of other applications that may be driver-sensitive.

2

Hello Sir 

Please do try to install the new driver and if you will keep having such issues try older versions of the driver or even a studio driver instead of game-ready one. 

3

Hello Copper,

also in order to reproduce and fix the issue, we need these information:

  1. The number of GPU driver and the way you have installed it (GeForce experience, via website, etc.).
  2. How many GPUs do you have?
  3. Which version of RealityCapture do you use? (You can find the version number once you click on blue question mark in the upper-right corner of the application).
  4. Which license of RealityCapture do you use?

Thank you.

4

Hello here is update to the following error.

Symptoms:CUDA errors during processing

Two immediate temporary solutions:

  1. Physically remove all GPU cards, except one - single card configuration.

or

  1. Install old GPU driver. The version 419.67 is working for us.

Reason:

The issue is with the CUDA memory de-allocation function, that has stopped working properly with latest NVIDIA GPU drivers.

More specifically the function CUDAFreeHost() resulted with success code, but the memory was not de-allocated and therefore after some time, the GPU pinned memory was filled up and the SW ended up with the message “CUDA error : 2 : Out of memory”.

It seems that this bug causes problems also to another CUDA developers:

https://devtalk.nvidia.com/…/cudafreehost-not-clearing-all…/

We have contacted NVIDIA and we hope that there will be a new driver with fix very soon.

Meanwhile, we are also trying to fix the problem by some workaround in our code and we will try to release hotfix ASAP.

Thanks to all, who has provided the details of the error. It has helped us to successfully re-produced the bug.