5.6 Path Tracing Crash / Increased VRAM Usage

As you are using the CVar r.RayTracing.Nanite.Mode 1, you should look at this post to resolve your VRAM usage.
You can see the VRAM usage with the tool Render Resource Viewer (menu Tools > Render Resource Viewer).