Why is Processing Mappings taking so long?

anonymous_user_1a4b6672 · October 6, 2015, 9:21pm

Hi All,

I am having problems with the Processing Mappings process of the Swarm Agent. Although the process starts with 6 cores after a short time, only one is processing. It also takes a lot of time to finish. Attached a screenshot of the Swarm Status and log. Anyone knows why is this?

Thanks you very much in advance. !

alt text

Log File Swarm agent:

17:44:56: [Interface:TryOpenConnection] Local connection established
17:44:56: [Job] Accepted Job 41FE5E1C-442412F6-95579A91-8F8620D8
17:44:57: [Job] Launched Job UnrealLightmass_2015-09-29_08-01-02_1325512-64bit.exe
17:44:57: [Job] PID is 383700
17:44:57: [Job] GUID is “41FE5E1C-442412F6-95579A91-8F8620D8”
17:44:57: LogLightmass:Display: Lightmass Win64 started on: QQ000CND41520TV. Command-line: 41FE5E1C442412F695579A918F8620D8
17:44:57: Lightmass Win64 started on: QQ000CND41520TV. Command-line: 41FE5E1C442412F695579A918F8620D8
17:44:57: LogLightmass:Display: Processing scene GUID: 41FE5E1C442412F695579A918F8620D8 with 6 threads
17:44:57: Processing scene GUID: 41FE5E1C442412F695579A918F8620D8 with 6 threads
17:44:57: Building static lighting…
17:44:57: LogGenericPlatformMisc:Warning: Failed to determine engine directory: Defaulting to …/…/…/Engine/
17:44:57: Failed to determine engine directory: Defaulting to …/…/…/Engine/
17:44:57: [Job] Found a parent connection for PID 383700
17:44:57: [Job] 743B84F6 → 685A4BFD
17:44:57: [Interface:TryOpenConnection] Local connection established
17:44:57: Measured CPU frequency: 2.69 GHz
17:44:57: FStaticLightingSystem started using GKDOPMaxTrisPerLeaf: 4
17:44:57: Number of texture mappings: 8
17:44:57: Number of fluid mappings: 0
17:44:57: Number of landscape mappings: 0
17:44:57: Number of BSP mappings: 0
17:44:57: Number of static mesh instance mappings: 8
17:44:57: Reserving memory for 8 meshes, 30533 vertices, 43408 triangles
17:44:57: Scene surface area calculated at 8.586 million units (99.871% of the estimated 8.597 million units)
17:44:57: Importance volume surface area calculated at 8.586 million units (0.000% of the estimated 0.000 million units)
17:44:57: Preallocated 0.0Gb for kDOP nodes and triangles
17:44:57: Building kDOP took 0.03 seconds.
17:44:57: Static lighting kDOP: 26420 nodes, 13211 leaves, 52844 triangles, 28325 vertices
17:44:57: Static lighting kDOP: 25.411% wasted space in leaves
17:44:57: kDopTree.Nodes : 2.8Mb
17:44:57: kDopTree.SOATriangles : 3.6Mb
17:44:57: kDOPTriangles : 0.0Mb
17:44:57: TrianglePayloads : 1.2Mb
17:44:57: MeshInfos : 0.0Mb
17:44:57: Vertices : 0.5Mb
17:44:57: UVs : 0.2Mb
17:44:57: LightmapUVs : 0.2Mb
17:44:57: Static lighting kDOP: 26420 nodes, 13211 leaves, 52844 triangles, 28325 vertices, 8.6 Mb
17:44:57: Processing…
17:44:57: EmitDirectPhotons complete, 0.048 million photons emitted in 0.1 seconds
17:44:57: EmitIndirectPhotons complete, 0.005 million photons emitted in 0.1 seconds
17:44:57: Marking Irradiance Photons complete, 0.004 million photons marked in 0.1 seconds
17:45:21: Caching Irradiance Photons complete, 0.390 million cache samples in 23.9 seconds
17:45:21: Calculate Irradiance Photons complete, 0.003 million irradiance calculations in 0.1 seconds
17:50:58: [Job] Job is a success!
17:50:58: Lighting 100.0%
17:51:07:
17:51:07:
17:51:07: Total Static Lighting time: 358.23 seconds, 6 threads
17:51:07: 0.0% 0.1s Scene setup
17:51:07: 0.0% 0.1s Emit Direct Photons
17:51:07: 0.0% 0.0s Cache Indirect Photon Paths
17:51:07: 0.0% 0.1s Emit Indirect Photons
17:51:07: 0.0% 0.1s Mark 0.004 million Irradiance Photons
17:51:07: 6.7% 23.9s Cache 0.390 million Irradiance Photon Samples on surfaces
17:51:07: 0.0% 0.1s Calculate 0.003 million Irradiance Photons
17:51:07: 100.0% 358.2s Lighting
17:51:07: 0.0% 0.0s Unaccounted
17:51:07:
17:51:07:
17:51:07: Total Irradiance Photon Caching thread seconds: 24.2
17:51:07: 24.1% 5.8s Octree traversal
17:51:07: 4.7% 1.1s 1.676 million Visibility rays
17:51:07: 71.2% 17.2s Unaccounted
17:51:07:
17:51:07:
17:51:07: Total busy Lighting thread seconds: 783.10
17:51:07: 2.0% 15.9s Texel and vertex setup
17:51:07: 29.7% 232.3s Direct lighting
17:51:07: 23.0% 179.8s Area shadows with 123.079 million rays
17:51:07: 0.6% 4.8s Signed distance field source sparse sampling
17:51:07: 1.1% 8.5s Signed distance field source refining sampling
17:51:07: 5.0% 38.9s Signed distance field transition searching
17:51:07: 0.0% 0.2s Unaccounted
17:51:07: 7.1% 55.4s Block on indirect lighting cache tasks
17:51:07: 4.9% 38.1s Block on IrradianceCache Interpolation tasks
17:51:07: 0.0% 0.0s Volume Sample placement
17:51:07: 27.6% 216.1s Unaccounted
17:51:07:
17:51:07:
17:51:07: Indirect lighting cache task thread seconds: 306.34
17:51:07: 4.7% 14.5s ImportancePhotonGatherTime
17:51:07: 0.0% 0.0s CalculateImportanceSampleTime
17:51:07: 68.6% 210.1s FirstBounceRayTraceTime for 83.583 million rays
17:51:07: 6.7% 20.6s CalculateExitantRadiance
17:51:07:
17:51:07: Traced 224.679 million first hit visibility rays for a total of 292.2 thread seconds (0.769 million per thread second)
17:51:07: Traced 1.819 million boolean visibility rays for a total of 1.4 thread seconds (1.336 million per thread second)
17:51:07: Scene radius 827.1, Importance bounds radius 0.0
17:51:07: 8 Mappings, 4.282 million Texels, 1.606 million mapped texels
17:51:07:
17:51:07: 43.7% of Total Lighting thread seconds on Texture Mappings, 0% on Volume Samples, 0% on Visibility, 56.3% Unaccounted
17:51:07: 28 Lights total, 76.7 Shadow rays per light sample on average
17:51:07: Signed distance field shadows: 6.7 average upsample factor, 2.904 million sparse source rays, 0.439 million refining source rays, 0.090 million transition search scatters
17:51:07: 27 Volume lighting samples, 100.0% placed on surfaces, 0.0% placed in the volume, 0.0 thread seconds
17:51:07:
17:51:07: 0.048 million first pass Photons Emitted (out of 0.048 million requested) to deposit 0.018 million Direct Photons and 42 Indirect Photon Paths, efficiency of 37.99%
17:51:07: 0.005 million second pass Photons Emitted (out of 0.005 million requested) to deposit 0.008 million Indirect Photons, efficiency of 147.10%
17:51:07: 0.346 million Photon Gathers, 0.390 million Irradiance Photon Gathers
17:51:07: 3.180 million Importance Photons found, 0.000 million Importance Photon PDF calculations
17:51:07: 19.9% Bounce 1 Irradiance cache miss rate (1.606 million lookups, 0.319 million misses, 0.067 million inside geometry)
17:51:07: Final Gather: 51.4s on 17.852 million base samples, 226.4s on 65.729 million refining samples for 3 refinement levels.
17:51:07: 6.7% due to brightness differences, 93.3% due to importance photons, Samples at depth: 25.0%, 33.1%, 41.9%,
17:51:07: 3213.8 Mb Peak Working Set
17:51:07:
17:51:07:
17:51:07: Lightmass on QQ000CND41520TV: 6:10 min total, 181 ms importing, 51 ms setup, 24.1 sec photons, 5:46 min processing, 0 ms extra exporting [8/8 mappings]. Threads: 40:22 min total, 5:42 min processing.
17:51:07: Lighting complete [Startup = 181 ms, Lighting = 6:10 min]
17:51:07: Failed to send back log file through Swarm!
17:51:07: [CloseConnection] Closing connection 685A4BFD using handle 685A4BFD
17:51:07: [CloseConnection] Connection confirmed for disconnection 685A4BFD
17:51:07: [CloseConnection] Connection disconnected 685A4BFD
17:51:07: [GetMessage] Safely returning to 685A4BFD with no message
17:51:07: [MaintainConnections] Local connection has closed (685A4BFD)
17:51:07: [MaintainConnections] Removed connection 685A4BFD
17:51:07: [CloseConnection] Closing connection 743B84F6 using handle 743B84F6
17:51:07: [CloseConnection] Connection confirmed for disconnection 743B84F6
17:51:07: [CloseConnection] Closing orphaned Job (41FE5E1C-442412F6-95579A91-8F8620D8)
17:51:07: [CloseConnection] Connection disconnected 743B84F6
17:51:07: [GetMessage] Safely returning to 743B84F6 with no message
17:51:08: [MaintainConnections] Local connection has closed (743B84F6)
17:51:08: [MaintainConnections] Removed connection 743B84F6
17:51:08: [MaintainConnections] All connections have closed
17:51:08: [Maintain Jobs] Killing rogue Job "41FE5E1C-442412F6-95579A91-8F8620

Elggetto · October 6, 2015, 9:52pm

If you go into Build → Lighting info → Lighting StaticMesh Info, is there a mesh that has a really big lightmap resolution?

anonymous_user_1a4b6672 · October 7, 2015, 7:13am

Hi, I have a mesh with a resolution of 2048… I supose that would qualify as a big lightmap?

anonymous_user_1a4b6672 · October 7, 2015, 7:18am

I changed it now, it is indeed much faster. Thanks!

Elggetto · October 7, 2015, 3:25pm

2048x2048 sould be considered a big lightmap.
For small/medium objects, usually nothing bigger than 250x250 is gonna be fine. Even 32x32 is good most of the time. If you have a really big object, consider breaking it in smaller parts.

It’s all about Lightmap Density!

Lightmap Density

What you can do is press alt-0 (or go into View Mode → Lightmap Density). You need to bake lighting first.

If it’s red, your resolution is probably too high. Blue, too low.
Aim for green!

However, if you’re doing ArchViz or something that needs to be realistic, you can go a bit higher with the values.

anonymous_user_1a4b6672 · October 8, 2015, 3:02pm

Thank you very much for the answer. You actually solved other questions that I had going on.

anonymous_user_e500d277 · January 6, 2018, 6:15pm

Really thanks, thanks!!!