Client Build takes to long

[dru-pruu]

Hello All,
CICD - Azure DevOps with self-hosted Build Agent with GPU.
Problem: before our DevOps did git checkout in separate step, like this:

git -C C:\CoSocietyUnReal\ pull -pt
git -C C:\CoSocietyUnReal\ checkout AI.Devops.Add_stages-to-pipline
git -C C:\CoSocietyUnReal\ pull -pt

And everything works fine, build (cook and build) took about 5-7 mins.

Right now we decided to move to classic Azure DevOps step - checkout:

          - checkout: self
            persistCredentials: true
            lfs: true

          - script: |
              git lfs fetch
              git lfs pull
            displayName: git-lfs

Theoretical, it should only change the build folder. I checked both folders (old and new) and they looks totally the same. But Each time, when I run pipeline - client step takes for me 1h +/-.

our build steps:

          - powershell: |
              & "C:\Program Files\Microsoft Visual Studio\2022\Community\dotnet\runtime\dotnet.exe" "C:\UnrealEngine\Engine\Binaries\DotNET\UnrealBuildTool\UnrealBuildTool.dll" -Target="MyProjectEditor Win64 Development -Project=$(Build.SourcesDirectory)\MyProject.uproject" -Target="ShaderCompileWorker Win64 Development -Quiet" -WaitMutex -FromMsBuild
            displayName: 'Client Compile Step'

          - powershell: |
              cmd.exe /c C:/UnrealEngine/Engine/Build/BatchFiles/RunUAT.bat  -ScriptsForProject=$(Build.SourcesDirectory)\MyProject.uproject BuildCookRun -project=$(Build.SourcesDirectory)\MyProject.uproject -noP4 -clientconfig=Development -serverconfig=Development -nocompileeditor -unrealexe=C:\UnrealEngine\Engine\Binaries\Win64\UnrealEditor-Cmd.exe -utf8output -platform=Win64 -build -cook -CookCultures=ru -pak -compressed -generatepatch -basedonreleaseversion="1.0.1" -stage -package -cmdline=" -Messaging" -addcmdline="-SessionId=XXX -SessionOwner=Administrator -SessionName=Patching_Client "
            displayName: 'Client Build & Cook'

I cannot understand why different git methods took so terrible build time effect. Any ideas? Thanks, Team!

[dru-pruu]

After I did some tests, I found the the problem somehow related to git (checkout).
I have multi-staging pipeline, something like this
stages:
stage1
checkout
server build

stage2
client build.

I disabled for test checkout step in the stage1 - and client build was fust (10 mins). After I enable it back - again long build for client. Checkout looks like this:
steps:

• checkout: self
  persistCredentials: true
  lfs: true

• script: |
  git lfs fetch
  git lfs pull
  displayName: git-lfs

Any ideas how to fix it? probably some caching? but what file should I cache?

[dru-pruu]

@dru-pruu

checkout: self
persistCredentials: true
lfs: true

solution was pretty easy:

• checkout: self
  persistCredentials: true
  lfs: true

but before it you need to clean up the folder for git.