Best p4Ignore for Populating Perforce with UE4 Engine Source?

For me I am having an issue with the provided ignore file.

# Ignore root Visual Studio solution files
/*.sln
/.p4sync.txt

# Ignore all Visual Studio temp files
*.suo
*.opensdf
*.sdf
/Engine/DerivedDataCache/*
**/DerivedDataCache/Boot.ddc
**/DerivedDataCache/**/*.udd

# Ignore all Intermediate and Saved directories
*/Intermediate/*
*/Saved/*
# Ignore UBT's configuration.xml
Engine/Programs/UnrealBuildTool/*
*.uatbuildrecord
*.tmp

# Ignore built binaries and temporary build files
*/obj/*
*.csprojAssemblyReference.cache

# Ignore UBT's log output files
/Engine/Programs/UnrealBuildTool/*.txt

# Ignore Python cached files
*.pyc

# Ignore JetBrain's IDE folders
.idea/
!.idea/runConfigurations
.gradle/

# Ignore autogenerated files from HoloLens WMRInterop
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/packages/*
/Engine/Source/ThirdParty/WindowsMixedRealityInterop/MixedRealityInteropHoloLens/Generated Files/*

What happens for me is that when I do p4 opened I get a list of files changed inside Intermidiate and Saved directories like so:

//ROOT/Intermediate/PipInstall/merged_requirements.in#2 - add default change (text)
//ROOT/Intermediate/PipInstall/pyreqs_plugins.list#2 - add default change (text)
//ROOT/Intermediate/ReimportCache/3688439234.bin#2 - add default change (binary)
//ROOT/Intermediate/ShaderAutogen/PCD3D_SM5/AutogenShaderHeaders.ush#2 - add default change (text)
//ROOT/Intermediate/ShaderAutogen/PCD3D_SM6/AutogenShaderHeaders.ush#2 - add default change (text)
//ROOT/Intermediate/TargetInfo.json#2 - add default change (text)
//ROOT/Saved/Config/CrashReportClient/UECC-Windows-3A6342AD45FE0BC72391ACB8D8633AE7/CrashReportClient.ini#1 - add default change (text)
//ROOT/Saved/Config/CrashReportClient/UECC-Windows-C4DFB2114E00044B7C38F287EC54FF7F/CrashReportClient.ini#1 - add default change (text)
//ROOT/Saved/Config/CrashReportClient/UECC-Windows-D4CEC832433E8B2F8FE581AF1CFC6C40/CrashReportClient.ini#1 - add default change (text)

and a lot of these all the time when switching.
Is there an issue with the ignore?
My workaround always when switching stream is to do a revert on these files but it is annoying…
Anyone encountered something like this?