UE 4.11 Hangs Cooking on Macbook Pro

mbehensky · April 6, 2016, 2:39pm

I’m just getting started with UE, working on VR development for the Gear VR. I started using 4.10.4 and had no problems building for either Android or Windows on either my desktop or my late 2008 Macbook Pro running Windows 10 under BootCamp (absolute top of the line for its day, with 8gb ram and a NVidia 9600M with 512mb video ram). Cooking was slow on the Macbook, but very usable given the limited capability of the Gear VR hardware.

When 4.11 was released I tried it out, and found that it always hung early in the cooking process on the Macbook. If you use the wizard to create a basic blank C++ project for mobile, it works fine if you just hit the “Play” button. If you try and “Launch” or Package, either on an android device or for windows, it hangs after the following entry in the log:

MainFrameActions: Packaging (Windows (64-bit)): UE4Editor-Cmd: [2016.04.06-14.08.49:253][ 0]LogTargetPlatformManager:Display: Building Assets For WindowsNoEditor
MainFrameActions: Packaging (Windows (64-bit)): UE4Editor-Cmd: [2016.04.06-14.08.51:586][ 0]LogCook:Display: Max memory allowance for cook 8192mb
MainFrameActions: Packaging (Windows (64-bit)): UE4Editor-Cmd: [2016.04.06-14.08.51:586][ 0]LogCook:Display: Mobile HDR setting 1

If you wait long enough (an hour or so with full processor utilization) it starts complaining about GPU timeouts. Ram usage is less than 50%.

It doesn’t work any better if you try to launch or package in OSX.

I understand that my Macbook Pro is far below the recommended minimum hardware for UE, and I don’t mind if it is slow, but its a shame if I can’t use it at all. It is also odd that it works in 4.10.4 but not 4.11.

I understand that allowing development on old hardware is probably not Epic’s highest priority, but its possible that this hang is a symptom of a larger problem. Given the GPU timeout complaints, it seems likely that the cooking process is doing some sort of computation on the GPU that doesn’t work on the 9600M.

Anyway, thanks for developing a great product. I can stick with 4.10.4 for now, or just use my desktop.

Here is some more information. If I use the Unreal Launcher, and do cook on the fly, I get a lot more logging of what is going on, and it gives an error instead of just hanging:

[2016.04.07-17.16.04:286][ 0]LogFileServer:Display: Read …/…/…/Engine/Content/Slate/Fonts/Roboto-Light.ttf, 162636 bytes
[2016.04.07-17.16.04:286][ 0]LogFileServer:Display: Read …/…/…/Engine/GlobalShaderCache-PCD3D_SM4.bin, 2131842 bytes
[2016.04.07-17.16.04:286][ 0]LogCook:Display: Cooking /Engine/EngineDamageTypes/DmgTypeBP_Environmental → C:/Users/maxb/Documents/Unreal Projects/Test2/Saved/Cooked/WindowsNoEditor/Engine/Content/EngineDamageTypes/DmgTypeBP_Environmental.uasset
[2016.04.07-17.16.04:327][ 0]LogSavePackage: Save=41.33ms
[2016.04.07-17.16.04:327][ 0]LogSavePackage: Async saving from memory to ‘C:/Users/maxb/Documents/Unreal Projects/Test2/Saved/Cooked/WindowsNoEditor/Engine/Content/EngineDamageTypes/DmgTypeBP_Environmental.uasset’
[2016.04.07-17.16.04:329][ 0]LogSavePackage: Total save time: 43.56ms
[2016.04.07-17.16.04:338][ 0]LogSavePackage:Display: Finished SavePackage C:/Users/maxb/Documents/Unreal Projects/Test2/Saved/Cooked/WindowsNoEditor/Engine/Content/EngineDamageTypes/DmgTypeBP_Environmental.uasset
[2016.04.07-17.16.04:345][ 0]LogCook:Display: Cook complete …/…/…/Engine/Content/EngineDamageTypes/DmgTypeBP_Environmental.uasset
[2016.04.07-17.16.04:345][ 0]LogFileServer:Display: Read …/…/…/Engine/Content/EngineDamageTypes/DmgTypeBP_Environmental.uasset, 960 bytes
[2016.04.07-17.16.04:346][ 0]LogLinker:Warning: Can’t find file ‘…/…/…/Engine/Content/L10N/en-US/EngineDamageTypes/DmgTypeBP_Environmental.uasset’
[2016.04.07-17.16.04:352][ 0]LogUObjectGlobals:Warning: Failed to load ‘…/…/…/Engine/Content/L10N/en-US/EngineDamageTypes/DmgTypeBP_Environmental.uasset’: Can’t find file ‘…/…/…/Engine/Content/L10N/en-US/EngineDamageTypes/DmgTypeBP_Environmental.uasset’
[2016.04.07-17.16.04:353][ 0]CookResults:Error: Error Error loading …/…/…/Engine/Content/L10N/en-US/EngineDamageTypes/DmgTypeBP_Environmental.uasset!

This is from a blank project with one sphere added. It looks like it cooks the environmental asset, but doesn’t create some file the next step expects.

Max Behensky

anonymous_user_e729f54e · April 13, 2016, 7:28pm

Hello ,

I have a few questions for you that will help narrow down what issue it is that you are experiencing.

Quick questions:

Could you provide your dxdiag?
Does this issue still occur in the 4.11.1?
What the project being used upgraded from an older version of the engine?

mbehensky · April 13, 2016, 11:52pm

I figured out what is going on last night. I rebuilt UE4.11.1 from source and started debugging it. After a lot of fooling around it became clear that the issue was the number of cores on the processor that the engine is running on. The cook always hung on a new basic project when running on a 2 core Intel core Duo, and always worked on a different machine with 4 cores and around the same processor vintage and memory.

I then tried using MSConfig to tell the 4 core machine it only had 2 cores, and then it hung on cook as well. I then patched the Windows NumCores and NumCoresIncludingHyperthreadsd functions to lie and say that the 2 core machine had 4 cores, and I was able to cook on the 2 core machine.

After more investigation I found the culprit, in the file Engine\Source\Runtime\Launch\Private\LaunchEngineLoop.cpp, in the function FEingineLoop::Preinit:

Here is the code:
#if WITH_EDITOR
// when we are in the editor we like to do things like build lighting and such
// this thread pool can be used for those purposes
GLargeThreadPool = FQueuedThreadPool::Allocate();
int32 NumThreadsInLargeThreadPool =
FPlatformMisc::NumberOfCoresIncludingHyperthreads() - 2;

	verify(GLargeThreadPool->Create(NumThreadsInLargeThreadPool));

#endif

As you can see, if the number of cores including hyperthreads is 2 or less, no threads are created in the large thread pool. This causes one of the tasks involved in cooking to never finish. I added two lines:

	if (NumThreadsInLargeThreadPool < 1)
		NumThreadsInLargeThreadPool = 1;

After this addition my two core machine cooks fine (although it is slow).

I have pushed these changes to my forked repository, and am going to do a pull request. I also found another problem that prevented the CookOnTheFly.cpp file from compiling if you set the DEBUG_COOKONTHEFLY flag to 1.

Thanks,

Max Behensky

mbehensky · April 13, 2016, 11:57pm

Wow, the formatting came out terribly there. Let me try doing the code again:

	// when we are in the editor we like to do things like build lighting

and such
// this thread pool can be used for those purposes

	GLargeThreadPool = FQueuedThreadPool::Allocate();

	int32 NumThreadsInLargeThreadPool =

FPlatformMisc::NumberOfCoresIncludingHyperthreads() - 2;

/////////////// Here is the fix
if (NumThreadsInLargeThreadPool < 1)
NumThreadsInLargeThreadPool = 1;
/////////////// End Fix

	verify(GLargeThreadPool->Create(NumThreadsInLargeThreadPool));

mbehensky · April 14, 2016, 12:06am

This is a repeat of the above comments, since I couldn’t get the formatting to come out right for the code patch…

I figured out what is going on last night. I rebuilt UE4.11.1 from source and started debugging it. After a lot of fooling around it became clear that the issue was the number of cores on the processor that the engine is running on. The cook always hung on a new basic project when running on a 2 core Intel core Duo, and always worked on a different machine with 4 cores and around the same processor vintage and memory.

I then tried using MSConfig to tell the 4 core machine it only had 2 cores, and then it hung on cook as well. I then patched the Windows NumCores and NumCoresIncludingHyperthreadsd functions to lie and say that the 2 core machine had 4 cores, and I was able to cook on the 2 core machine.

After more investigation I found the culprit Here is the code fix again in the in the file Engine\Source\Runtime\Launch\Private\LaunchEngineLoop.cpp, in the function FEingineLoop::Preinit:

	// when we are in the editor we like to do things like build lighting and such
	// this thread pool can be used for those purposes
	GLargeThreadPool = FQueuedThreadPool::Allocate();
	int32 NumThreadsInLargeThreadPool = FPlatformMisc::NumberOfCoresIncludingHyperthreads() - 2;

//////////////// Here is what I added. Note that if NumberOfCoresIncludingHyperthreads() returns 2 or less,

//////////////// NumThreadsInLargeThreadPool will be zero or negative. The patch below makes sure you always
//////////////// have at least one thread.

	if (NumThreadsInLargeThreadPool < 1)
		NumThreadsInLargeThreadPool = 1;

//////////////// End of patch

	verify(GLargeThreadPool->Create(NumThreadsInLargeThreadPool))

I’m going to do a pull request to try and get this incorporated into the next release…

vibgyorc6 · April 14, 2016, 12:20am

Wow, thank you so much for the fix! I’ve been having the same problem ever since the 4.11 preview and it’s been causing me loads of grief since I accidentally upgraded my project permanently and couldn’t cook my project. Thanks to you it’s finally working again!

MrMagarac · April 15, 2016, 9:49am

I have the same issue when trying to package any template. It hangs forever after these lines:
Max memory allowance for cook 8192mb
Mobile HDR setting 1

I also use 2 core system (Intel Celeron G1840) but i didn’t have that issue on 4.10, i could package any tamplate in under a minute. I don’t know if this would help my case since i have no idea how to patch this thing but i hope next engine update will have this fix in place so i could try it out! =)

mbehensky · April 15, 2016, 6:51pm

Its not that hard to do the patch. First, you have to get set up to build the unreal engine from source. There is good documentation on how to do this, although some of it is a little out of date. You should use Visual Studio 2015 (the community edition is free).

Here are some notes on my experience doing this. Once you have checked out the source repository from GitHub, run setup.bat in the top level unreal engine directory. Next, open a command window and navigate to the Unreal Engine directory. Run:

GenerateProjectFiles -2015

in the unreal directory (there is a space before the -2015). If you don’t include the -2015 it makes project files for visual studio 2013, which is probably not the visual studio version you want.

Next, open the UE4.sln file in the unreal directory. Select Development-Editor in the configuration pull down box. Do Build->Build Solution, and go out for a walk (it will take a really long time).

After that finishes, select Development in the configuration pull down box and build that too. It will also take a long time.

You supposedly need to register your new engine by running UnrealVersionSelector-Win64-Shipping.exe in the Engine\Binaries\Win64 directory.

Go back to the Development-Editor configuration and do Debug->Start Debugging and make sure the engine works.

If everything seems good, try the patch. You can find the LaunchEngineLoop.cpp file in the solution explorer in UE4->Source->Runtime->Launch->Private.

Find the lines before the patch, and edit the patch in. Build both the Development and Development-Editor configurations (it will go a lot faster the second time). Hopefully you will be able to cook your game in 4.11 at that point.

Good Luck…

MrMagarac · April 17, 2016, 5:54pm

I’ve followed all those steps and i still can’t package projects, stuck on LogCook:Display: Mobile HDR setting 1.

It took a while to dl everything, build, test and so on, i really had no idea how long it would take for all that. But engine is up and running and looks like everything is working except packaging of course.

When building dev-editor: =Build: 20 succeeded, 0 failed, 1 up-to-date, 27 skipped=
Development: =Build: 24 succeeded, 1 failed, 18 up-to-date, 5 skipped=

I think this is what failed:
Engine\Intermediate\Build\Win64\MinidumpDiagnostics\Development\EditorStyle\Module.EditorStyle.cpp : warning C4653: compiler option ‘Optimizations (one or more of /Oawstgp[y]) or debug checks (one or more of /GZ, /RTCcsu)’ inconsistent with precompiled header; current command-line option ignored
17>d:\unreal clone\unrealengine\engine\source\editor\editorstyle\private\slateeditorstyle.cpp(159): error C4883: ‘FSlateEditorStyle::FStyle::SetupGeneralStyles’: function size suppresses optimizations
17> -------- End Detailed Actions Stats
17>ERROR : UBT error : Failed to produce item: D:\Unreal Clone\UnrealEngine\Engine\Binaries\Win64\MinidumpDiagnostics.pdb
17> Total build time: 68,25 seconds

Ignored that and went to debugging, engine launched, picked one template, shaders compiled and i can play. I’ve put code in, built both solutions, still get that one fail in (Development) not in (dev-editor) build. Then tried debugging again, engine launched, picked template and tried package. First it took like 40 mins or more to compile something, like 190 things, then it went to Mobile HDR setting 1 and no go from there.

Tried to package template again, this time it went to “mobile hdr” in 30sec or so and stopped, 15min in still not building.

Now i don’t know if that one fail is preventing code to fully build (it did say build failed in lower left corner of VS) or did i mess up something in the process.

Anyway, thank you for trying to help me i really appreciate it! =)

mbehensky · April 19, 2016, 5:42pm

Hmm. I don’t know what is up. You need to address that build error somehow, I bet. Is your Visual Studio 2015 up to date?

I would also make sure that you are really using your version that was built from source. When I was working on this, I removed all my unreal versions from the launcher, and then uninstalled the launcher so I could be absolutely sure that I was using the version I was building.

Make a desktop shortcut that points to the unreal engine in \Engine\Binaries\Win64\UE4Editor.exe so you can run your new version easily.

mbehensky · April 19, 2016, 6:00pm

Also, when I did this the unreal version was release 4.11.1. Are you on the release branch in git?

It looks like they just updated to 4.11.2 a few hours ago. I’m going to check it out now and build to see if they incorporated my fix, and if not make sure it still works if I put the patch back in.

anonymous_user_2ef4fb84 · April 20, 2016, 8:25am

The same thing(( Can’t compile any project 4.11 - 4.11.2 Example log of the standard ThirdPerson (Blueprint)
[ThirdPerson.log][1]

mbehensky · April 20, 2016, 3:27pm

All right, I’ve tried out 4.11.2 and I can confirm that the stock version will NOT cook successfully on my dual core MacBook Pro.

I checked out the source for 4.11.2 and they have not incorporated my patch yet. A few days ago, I did get the following message from robertmanuszewski with regard to my pull request, and they are working on it:

"Thanks, !

I already merged this into one of our Dev streams."

I merged my patch into the 4.11.2 source and rebuilt it (on my quad core, it goes faster). I had no compile errors.I then copied all the newly built .exe files from Engine\Binaries\Win64 into the Official 4.11\Engine\Binaries\Win64 directory on my dual core. This works better than trying to run out of the source directory, since it takes advantage of a bunch of pre-built assets, and lets you use the unreal launcher.

I checked, and the .exe files are the only thing that change when you add my patch and rebuild. The .dll files don’t get re-linked.

Anyway, using these executables, I just created a new blank c++ project and it cooked just fine in a couple of minutes on my dual core.

anonymous_user_31c206b8 · April 21, 2016, 4:32pm

This is due to a new compiler warning in Visual Studio 2015 Update 2. I’ve had a it trigger on automatically generated code for the Unreal editor (the function was over 1,700 lines long).

You can disable the warning that leads to the error with:

#pragma warning(disable: 4883)

This is going to be a very common issue with Unreal users soon…

DanielWhite2015 · May 17, 2016, 12:48pm

hi there, I’m having the same issue and am currently downloading the 4.11 branch of the soruce code - has your patch been implemented as yet? if not would you be able to send me the aptch so I can try it on my AMD dual core?

anonymous_user_2ef4fb84 · May 22, 2016, 11:25am

Me too please)

anonymous_user_564a133f · May 24, 2016, 1:15pm

Thanks man!. My core 2 duo works again with latest version of engine!. 4.10 is the last version that works without patch. This patch solves error “TEXTURE STREAMING POOL OVER…” also (in 4.10.2 version)!. I tested in Mac and Windows.

Notes: Build configuration “Development-editor” isn’t necessary. In mac, is necessary build “ShaderCompilerWorker” also (and UE4).

Nicate · June 2, 2016, 11:08pm

's patch has been included in Unreal 4.12.0, addressing the issue. I can now finally package again, without also needing Visual Studio (which on my machine is unfortunately currently not possible). Thanks to all involved! Especially the patch author