OK Now add stuff back to the HLOD in batches, test HLOD building each time, don’t worry about using a new HLOD layer, just use the same one.
When it stops working whatever you added last has the problem in it and then you can narrow down your search.
The virtual memory is located for Windows 10, in Advanced System Settings in the Performance Section, Advanced tab.
Look for Virtual memory, then change the settings to suit, let me know if you need more help with that