This sounds very interesting, care to share more?
Hmm not quite. Actually you would have to run it on a single machine that has a single core CPU.
Otherwise you still get the seams as objects are still rendered in different threads… 
Nope, I mean I’ll change the algorithm so that it doesn’t produce any seams when run on a single machine.
Yeah, the source is available if someone wants to modify Lightmass to avoid the issue.
That same algo could be used then to make it seamless even across a swarm cluster. Why would it be limited to a single machine?
I also have some ideas for a better solution than what lightmass currently implements.
But Im a bit too weak on the C++ side to do that alone…
The problem is the algorithm requires data transfer between threads/machines. The first one can be done relatively easily, while the second one (global all-to-all communication) is not even supported by Swarm at all…
P.S.: Swarm is the root cause of many “bad” designs of current Lightmass. It just grabs jobs from the editor, estimates their costs and distributes them to machines, and finally gets them back. No data transfer is supported in the middle of a lighting build, no global synchronization - it lacks many common parallel computing primitives. If we had them then we can solve many problems such as highly uneven distribution of large ( >1k ) lightmaps, seams and so on. However I’m not strong in parallel computing, I can only hope someone else can do this.
I actually thought about redesigning swarm as well. 
This would be a main feature of my version.
That would be REALLY helpful if you can do it! Maybe you can share your early progress somewhere when you start it
So, I’m not nearly as experienced a coder as you and Kvogler, but something did come to mind to me that perhaps might be a decent temporary solution.
Have you guys tried making lightmass operate in a virtual machine type scenario? The only reason I ask is because when I was younger I would mess around with custom Linux builds and VirtualBox and the like. And there ways that you workaround multi-core requirements on a single-machine by fooling the vurtualized OS into believing that the system was a dual-core instead. Of course, that wasn’t without its own drawbacks. Like the overall performance dropped by like 60-70% and numerous bugs, at least in my case…
Anyway, is it possible to virtualize Lighmass’ process and then do the inverse of what I mentioned above? So spoof the virtualized environment into believing that the multi-threaded CPU is actually just a beefy single threaded processor?
Also, is there anyway we can disable multi-threaded lightmass calculation on our own? Because I have a machine dedicated specifically for lightmass baking and nothing else, so if I can get better results and all I need to do is wait longer, I’m completely game. (I’m asking because I’m afraid if I go in without already knowing what I’m looking, I’m going to completely break everything.)
Nah Im really a noob when it comes to C++. Let alone the unreal flavored version.
Limiting the lightmap calculations forcibly to a single thread is like cutting off your leg because your ankle hurts.
I would know of a perfect way to achieve uniform lighting quality across a scene, but as I said before, Im too weak at C++ to do that.
So would you like to elaborate a little bit on your idea?
Hmkay
Just to get the picture (and a chance to correct me if Im wrong), currently lighmass computes lighting, ignoring swarm here, like this:
- Take a list of all mesh objects to include (scene data)
- Each item is processed by an individual thread
The thread does:- Calculate the direct lighting.
- Calculate the contributed indirect lighing from other meshes in the scene
- Repeat previous step as per number of bounces
- Provide results for gathering
The problem is now that when a thread calculating lighting for mesh A and checks for indirect lighting data from another mesh B, it depends on how far other threads have progressed with their bounce calculations that indfluence mesh B.
Thus, mesh A nd B might be subjected to a similar environment and no seam is expected, but this kind of race condition makes the calculation off.
My idea would be to make the multi threading bounce based, not mesh based.
As a high level sketch:
- Take a list of all mesh objects to include (scene data, like above)
- Derive “Bounce lists”. One for each bounce, which just describes ray geometry in terms of surfaces, a bit like light transport notation (see below).
- The bounce lists are processed in sequence, starting with bounce zero (direct lighting) and then all subsequent bounces
- Each item of a bounce list is processed by a thread that uses the LTN data and surface information to calulate the lighting
The key point here is that all calculations for a specific bounce are completed before any next bounce is calculated.
The order in which the threads process the items of a particular bounce list is irrelevant, thus not creating artefacts
Just my 2 cents on this.
Keep in mind: Its a very high level description. There are some more details to work out.
Cheers,
Klaus
Do you have a background in computer graphics, especially have understanding about classical rendering algorithms like path tracing/bidirectional path tracing, photon mapping or radiosity? 
I once wrote my own little rendering program (similar to POV ray). So I kind of got the hang of it.
My lightmapping would be very similar to classical raytracing.
I just cant figure out how Unreal organizes, repacks and atlases the UVs for the encoded lightmaps and how the encoding works in detail.
Also I dont understand how lightmass accesses scene geometry, etc…
Oh that is cool. Then I think I can point out the root cause of the seams to you and you should be able to understand it. They are produced by parallel irradiance caching without a final merge. (which means it is not caused by race condition between bounce calculations, they are actually on the same bounce, if you had taken a look into lightmass. But I don’t want to let you down so I didn’t tell you this at the beginning)
P.S.: This can be easily confirmed by setting bAllowIrradianceCaching to false in BaseLightmass.ini. Lighting build will be much slower, but still multithreaded, without any seams.
Ah ok then. Dont worry. Standing corrected doesnt let me down.
Always enjoying learning
If you go into the command console line and type:
r.ssr.quality 4
the number of bounces will help - not sure why people say the number of bounces won’t help when clearly they do.
SSR (screen space reflection) has nothing to do with this at all, this is about the lighting.
This is an issue with the indirect lighting samples, the number of bounces will not improve the splotches or the lighting variations.
So how much slower are we talking here? Or is it not really a predictable increase of computation time?
I see… And why is that?