Level Streaming failing on 4.15 - MAC & iOS packaged, Cooked PC

Development World Creation
, , , , ,
1

Update

  • Builds that work - PIE PC and Mac, packaged for distribution PC
  • Builds that fail - Packaged for distribution MAC, iOS, Cooked DebugGame on PC

This is probably user error, but it’s code that’s worked since 4.4ish and works on PC (including packaged builds).

My game has multiple minigames. Minigames can specify a level to load by name (stored in some json). Levels are stored as sublevels on the base level that is loaded.

They are then loaded through C++ or blueprint. Or not loaded in the case of 4.15…

Error seen:

LogStreaming:Error: Couldn't find file for package /Game/Levels/Speedy/SpeedyLevel.

That trailing ‘.’ is pretty suspicious. I haven’t told it about Levels/Speedy - just using the level name as seen in the levels window to specify what to load.

Side note: I also notice that the core level takes 1.6 seconds to load despite it containing only a single camera. Excessive?

I’ve tried a couple of approaches to making it work - 1) The original (probably nearly two years of just working ™)

           ULevelStreaming *pStreaming = UGameplayStatics::GetStreamingLevel(pWorld, LevelToLoad);
			if (pStreaming)
			{
                
				pStreaming->bShouldBeLoaded = true;
				pStreaming->bShouldBeVisible = m_bShowRequested;
				LevelVerbosef(_T("FLevelLink::Load loading '%s'"), *LevelName);
			}

And also:

FLatentActionInfo LatentInfo;
UGameplayStatics::LoadStreamLevel(mp_World.Get(), LevelToLoad, m_bShowRequested, false, LatentInfo);

If I right click on the SpeedyLevel and get a reference then I get this:

World'/Game/Levels/Speedy/SpeedyLevel.SpeedyLevel'

Looking in the iOS cookeddata folder the umap (and other data) exists as you’d expect.

As far as I can tell PC works correctly, though now I’m going to dive back in and see if I can force the same error.

2

Seems like PC is fine. I killed all the intermediate / cooked / saved data I could find and it still works when repackaged.

I tried the option to package all assets within my level folder. This increased package size, but we’re still not streaming.

I’ve also added a little blueprint reference of the streaming levels (putting them in an array) in case this “helped” the system know it needs to reference the levels.

Still not streaming.

3

Hmmmm.

When I load a level via blueprint C++ subsequently works however once C++ unloads the level it fails the next time it tries to load it.

Perhaps this isn’t a packaging problem, but an identification problem?

4

When I try streaming in “SpeedyLevel” from the Event Tick in my persistent level I get the following assertion:

2017-05-02 10:10:12.464533-0700 MM_04[4032:332105] Assertion failed: 0 [File:/Users/build/Build/++UE4+Release-4.15+Compile/Sync/Engine/Source/Runtime/CoreUObject/Private/Serialization/AsyncLoading.cpp] [Line: 1623]

Which is here:

 // @todoio: why do we need this? some UFunctions have null outer in the linker.
    		if (Import->ClassName != NAME_Package)
    		{
    			check(0);
    			continue;
    		}

This works when not packaged (level can be streamed in and out repeatedly).

5

However referencing the level by /Game/Levels/Speedy/SpeedyLevel works on iOS via blueprint (I can load / unload the level repeatedly).

So - I think there is a bug in that iOS / Mac packaged behaves differently to PC packaged / the editor.

6

Red herring encountered above!

Working theory now is that level streaming breaks after I mount a pak file.

7

FCoreDelegates::OnMountPak.Execute(rPakFileName, 0, nullptr)

After this level streaming breaks.

8

I’m at a loss.

I looked at the hotfix code and it just seems to OnMountPak in the same way.

I’ve also tried using FPakPlatformFile with the same result.

I can load data from the pak post mount, but have obviously broken some link in the file system.

This is day three of poking around here, in a pretty random way so any help would be appreciated. Any ideas on who would know how to mount a pak file in 4.15? Or where to look for documentation on how to do this properly?

@joeGraf @muzaheed @Rama @Alex4DPOC

9

Its pretty clear from poking around in the forums and here on answerhub that a lot of user time is being wasted with this question of how to mount pak files. From what I’ve seen online I’d estimate that months of programmer time has already gone down this rabbit hole. I’ve spent a couple of weeks myself, and am currently stuck and considering a return to 4.14 (which will then mean downloading engine code and fixing it to work with iOS 8.3 so even that approach will probably cost another day).

While there are bits of individual documentation here and there there seems to be no single point of reference for the [should be simple] task of how to mount and unmount a pak file correctly.

Please can Epic address this…

More than slightly frustrated.

10

in 4.14 and older version pak mounting should work for all platforms. at least working on my side
in 4.14 apic has introduced pak mounting from blueprint. u can try that one and should work

11

Returning to 4.14 is an option I may have to pursue, however this seems like something that should be fixed!

Just confirmed the same behavior in 4.16

Will look into pak file mounting from blueprint.

12

And I got this to happen on PC. Cooked content and running the DebugGame from VS. Interesting that it didn’t happen in a purely packaged build.

13

Hey theonecalledtom,

Since you’ve been able to reproduce the issue on multiple platforms, would you be able to provide us with a repro case that we can use to test it on our end? I’m not sure what your setup looks like, so if you’re able to provide me with a list of steps I can follow or a simplified test project I’d be glad to take a look and see what I can find.

Thanks!

14

will do!

Please find an attached zip that demos the problem.

Steps to repro:

Setup

  1. Extract zip, open build as needed
  2. Place the contained “test.pak” file at c:\ - if this isn’t suitable you can edit the location in the level blueprint for BaseMap, under the Left Mouse Click event.

Working behavior (during PIE).

  1. Run
  2. Left mouse click toggles a blue sphere in a streamed level
  3. Right mouse click toggles the pak mount (nothing visual)
  4. Left mouse click will continue to work

Not working test (Cooked)

  1. Cook assets for PC (or MAC).
  2. Create Visual Studio project files
  3. Open VS, set build to DebugGame
  4. Repeat the above test, note that after the right click, the left click stops working.

And I just verified that like in my real project, the packaged PC build works.

link text

15

For cooked data on PC it looks like the lower level isn’t being consulted here (IsNonPakFilenameAllowed returns false):

virtual FDateTime GetTimeStamp(const TCHAR* Filename) override
	{
		// Check pak files first.
		FPakFile* PakFile = NULL;
		if (FindFileInPakFiles(Filename, &PakFile) != NULL)
		{
			return PakFile->GetTimestamp();
		}
		// Fall back to lower level.
		FDateTime Result = FDateTime::MinValue();
		if (IsNonPakFilenameAllowed(Filename))
		{
			double StartTime = (GNewAsyncIO && UE_LOG_ACTIVE(LogPakFile, Verbose)) ? FPlatformTime::Seconds() : 0.0;
			Result = LowerLevel->GetTimeStamp(Filename);
			UE_CLOG(GNewAsyncIO, LogPakFile, Verbose, TEXT("GetTimeStamp on disk (!!) for %s took %6.2fms."), Filename, float(FPlatformTime::Seconds() - StartTime) * 1000.0f);
		}
		return Result;
	}

So the problem here is that there is an expectation of not mixing pak files with loose files. My guess is this is a separate issue to my iOS / Mac problems which also occur in completely packaged builds.

First I looked into flipping the relationship around, so the FPakPlatformFile goes underneath but it’s engineered to expect to be above the base platform file system.

16

Then I thought that enabling EXCLUDE_NONPAK_UE_EXTENSIONS might be at least a stop gap solution. However if you look at the callstack below - we’re using FindPackageFileWithoutExtension so that option doesn’t work.

	PakFileMountTest-Win64-DebugGame.exe!FPakPlatformFile::GetTimeStamp(const wchar_t * Filename)  Line 1175	C++
 	PakFileMountTest-Win64-DebugGame.exe!FFileManagerGeneric::GetTimeStamp(const wchar_t * Filename)  Line 489 + 0x6b bytes	C++
 	PakFileMountTest-Win64-DebugGame.exe!FPackageName::FindPackageFileWithoutExtension(const FString & InPackageFilename, FString & OutFilename)  Line 626	C++
>	PakFileMountTest-Win64-DebugGame.exe!FPackageName::DoesPackageExist(const FString & LongPackageName, const FGuid * Guid, FString * OutFilename)  Line 668 + 0xd bytes	C++
17

Actually EXCLUDE_NONPAK_UE_EXTENSIONS does help - I read the logic backwards. However I don’t seem to be able to set it in a x.build.cs file, but have to set it in IPlatformFilePak.cpp itself.

So thats a potential fix with an engine mod.

Personally I’d prefer to just have a flag in the FPakPlatformFile, so we could allow for a couple of them in the file stack with different behaviors if needed.

And now I have to move my slow ■■■■■ mac over to an engine source build to see if this fixes the problem there too.

Epic owe’s me an SSD for the machine… or a beer.

18

Confirmed that the solution works on iOS too.

My suggested fix (for the engine) is to add the following function to FPakPlatformFile:

/**
	 * Sets up the list of file extensions that this pak file layer doesn't fallback to the lower level for
	 *
	 * @param File extensions that we don't bother looking in the lower levels for
	 */
	void SetLowerLevelExclusions(const TArray<FName> &exclusions)
	{
		ExcludedNonPakExtensions.Reset();
		ExcludedNonPakExtensions.Append(exclusions);
	}

And to modify the exclusion code to (not required, seems good though):

bool FPakPlatformFile::IsNonPakFilenameAllowed(const FString& InFilename)
{
	bool bAllowed = true;

#if EXCLUDE_NONPAK_UE_EXTENSIONS
	if ((PakFiles.Num() || UE_BUILD_SHIPPING) && ExcludedNonPakExtensions.Num())
	{
		FName Ext = FName(*FPaths::GetExtension(InFilename));
		bAllowed = !ExcludedNonPakExtensions.Contains(Ext);
	}
#endif

I would also suggest moving the setup of ExcludedNonPakExtensions currently found in FPakPlatformFile::Initialize to a higher level place, but if the ability to set them externally is available this is a bit moot, just it’s pretty hidden the way it is.

Another option might be some first time only output as each extension gets excluded.

19

If you have a suggested fix, you are more than welcome to create a pull request:

https://github.com/EpicGames/UnrealEngine/compare?expand=1

This will allow our developers to take a look at the fix, and if the fix gets accepted you will be credited for your contribution.

20

Pull request: Fix for bugs in projects using FPakPlatformFile to mount DLC pak files. #3548

Maintaining engine source for this one small change is some serious friction. I can’t tell you how much I’d appreciate words going to the right ears so it can be pulled in and distributed before 4.17 is released. I’m likely going to be stuck on 4.16 for a while as 4.17 looks likely to drop iphone support.