Virtual Asset's fail to submit if there is more than 8 files

TristonS · December 31, 2024, 2:30am

Virtual Asset’s fail to submit if there is more than 8 files

I have tried streams, standard depots, setting connections to -1, 8, 100, batch size to 1, 5, 8, 100

I have tried submitting one asset, two assets, three assets . . . . and it all works perfectly, until there is more than 8.

I even tried compiling the UEVirtualAsset tool and using CLI, this tool works, but not if you have more than 8 assets in your submission

If you submit 9 asset’s in a single changelog with p4 backend, it will always fail, as it deletes the asset’s AFTER they are “added” but BEFORE they are submitted, resulting in a librarian error.

This is really strange behaviour, confirmed in 5.3 and 5.4

I’ve read the source code, seen the additional parameters, etc. but nothing has a positive effect in improving this behaviour.

Any insights would be really appreciated! Thanks! You can see in my log file, submissions working until I try a submission with 8 assets. Doesn’t seem to matter how many need to be virtualized, i.e. if you submit 10, and 5 are already virtualized, it will fail for all 10 the same way.

(Simplified)

[Core.ContentVirtualization]
SystemName=Default
 
[Core.VirtualizationModule]
BackendGraph=VABackendGraph_Example


[VABackendGraph_Example]
PersistentStorageHierarchy=(Entry=SourceControlCache)
SourceControlCache=(Type=p4SourceControl, DepotRoot="//ue_virtual",IgnoreFile=".p4ignore")

LogSlate: Window 'Submit Files' being destroyed
LogSourceControl: Attempting 'p4 fstat -Or U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/1x1x1_Box_Pivot_-XYZ.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/1x1x1BoxCenterAligned.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/AnimTreeEd_PreviewFloor.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ArcadeEditorSphere.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/Axis_Guide.uasset U:/environment_testrun/Unreal_Projects/Enviro
nments/NewProject_Test/Content/BackgroundCube.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BackgroundCube1.uasset'
LogSourceControl: Attempting 'p4 diff -sa U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/1x1x1_Box_Pivot_-XYZ.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/1x1x1BoxCenterAligned.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/AnimTreeEd_PreviewFloor.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ArcadeEditorSphere.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/Axis_Guide.uasset U:/environment_testrun/Unreal_Projects/Environ
ments/NewProject_Test/Content/BackgroundCube.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BackgroundCube1.uasset'
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/1x1x1_Box_Pivot_-XYZ.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/1x1x1BoxCenterAligned.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/AnimTreeEd_PreviewFloor.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ArcadeEditorSphere.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/Axis_Guide.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BackgroundCube.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BackgroundCube1.uasset - file(s) not opened for edit.
LogVirtualization: Display: Considering 7 file(s) for virtualization
LogVirtualization: Display: Found 7 package(s), 7 of which had payload trailers
LogVirtualization: Display: Found 7 locally stored payload(s) in 7 package(s) that maybe need to be virtualized
LogVirtualization: Display: Pushing payload(s) to cached storage is disbled, skipping
LogVirtualization: Display: Pushing payload(s) to EStorageType::Persistent storage...
LogVirtualization: [P4SourceControl - SourceControlCache] Found 7 unique payload(s)
LogSourceControl: Attempting 'p4 fstat -Or //ue_virtual/2f/8b/89/a6173e934c8d702d5a68a87ac352c4981d.upayload //ue_virtual/bd/14/48/043e65641c9bf6d1b3425364b48e37e524.upayload //ue_virtual/2e/34/6b/ed882e97be72dd074369054f501d978907.upayload //ue_virtual/ce/8a/26/5bb3d925ed56fc0c8985d86aea9df1ef46.upayload //ue_virtual/61/84/c8/d2bb8421c5943709bfcfa264ff447de5f2.upayload //ue_virtual/9f/81/77/29642f2e3100dba269ccfe356047a62dfb.upayload //ue_virtual/c4/42/f2/23792900102f0098b34998aff75fe3e4cc.upayload'
LogVirtualization: [P4SourceControl - SourceControlCache] Determined that 7 payload(s) require submission to revision control
LogVirtualization: [P4SourceControl - SourceControlCache] Started payload submission session '65F27D0E4409AD2B2C8BE3994119A63E' for '7' payload(s)
LogVirtualization: [P4SourceControl - SourceControlCache] Created directory 'U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e' to submit payloads from
SourceControl: CommandMessage Command: CreateWorkspace, Info: Client VASubmission-65f27d0e-4409-ad2b-2c8b-e3994119a63e saved.
LogSourceControl: Switched workspaces from '' to 'VASubmission-65f27d0e-4409-ad2b-2c8b-e3994119a63e%'
LogVirtualization: [P4SourceControl - SourceControlCache] Splitting the submission into '1' batches
LogVirtualization: [P4SourceControl - SourceControlCache] Processing batch 1/1...
LogSourceControl: Attempting 'p4 add U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/2f/8b/89/a6173e934c8d702d5a68a87ac352c4981d.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/bd/14/48/043e65641c9bf6d1b3425364b48e37e524.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/2e/34/6b/ed882e97be72dd074369054f501d978907.upayload U:/environment_testrun/Unreal_Pr
ojects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/ce/8a/26/5bb3d925ed56fc0c8985d86aea9df1ef46.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/61/84/c8/d2bb8421c5943709bfcfa264ff447de5f2.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/9f/81/77/29642f2e3100dba269ccfe356047a62dfb.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-
ad2b-2c8b-e3994119a63e/c4/42/f2/23792900102f0098b34998aff75fe3e4cc.upayload'
LogSourceControl: Attempting 'p4 reopen -c 1322 U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/2f/8b/89/a6173e934c8d702d5a68a87ac352c4981d.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/bd/14/48/043e65641c9bf6d1b3425364b48e37e524.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/2e/34/6b/ed882e97be72dd074369054f501d978907.upayload U:/environment_testru
n/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/ce/8a/26/5bb3d925ed56fc0c8985d86aea9df1ef46.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/61/84/c8/d2bb8421c5943709bfcfa264ff447de5f2.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f27d0e-4409-ad2b-2c8b-e3994119a63e/9f/81/77/29642f2e3100dba269ccfe356047a62dfb.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/65f
27d0e-4409-ad2b-2c8b-e3994119a63e/c4/42/f2/23792900102f0098b34998aff75fe3e4cc.upayload'
LogSourceControl: Attempting 'p4 submit -c 1322'
SourceControl: CommandMessage Command: CheckIn, Info: Change 1322 created.
LogSourceControl: Switched workspaces from 'VASubmission-65f27d0e-4409-ad2b-2c8b-e3994119a63e' to '%'
LogSourceControl: Attempting 'p4 client -d VASubmission-65f27d0e-4409-ad2b-2c8b-e3994119a63e'
SourceControl: CommandMessage Command: DeleteWorkspace, Info: Client VASubmission-65f27d0e-4409-ad2b-2c8b-e3994119a63e deleted.
LogVirtualization: Display: Pushed 7 payload(s) to EStorageType::Persistent storage
LogVirtualization: Display: 7 package(s) had their trailer container modified and need to be updated
LogVirtualization: Display: Detaching loaded packages from disk...
LogVirtualization: Display: Reset the loaders of 7 package(s)
LogVirtualization: Display: Checking packages for write access permission...
LogVirtualization: Display: All packages have write permission
LogVirtualization: Display: Replacing old packages with the virtualized version...
LogVirtualization: Display: Total Reduction 162.165 KiB (267.243 KiB -> 105.078 KiB)
LogVirtualization: Display: Replaced 7 package(s)
LogVirtualization: Display: Virtualization process complete
LogSourceControl: Attempting 'p4 reopen -c 1323 U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/1x1x1_Box_Pivot_-XYZ.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/1x1x1BoxCenterAligned.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/AnimTreeEd_PreviewFloor.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ArcadeEditorSphere.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/Axis_Guide.uasset U:/environment_testrun/Unreal_Projects/E
nvironments/NewProject_Test/Content/BackgroundCube.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BackgroundCube1.uasset'
LogSourceControl: Attempting 'p4 submit -c 1323'
SourceControl: CommandMessage Command: CheckIn, Info: Change 1323 created.
SourceControl: Submitted changelist 1323
LogAutomationController: Ignoring very large delta of 4.05 seconds in calls to FAutomationControllerManager::Tick() and not penalizing unresponsive tests
LogSlate: Window 'Submit Files' being destroyed
LogSourceControl: Attempting 'p4 fstat -Or U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BoundingBoxCorner.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BoundingBoxEdge.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/Cone.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow2_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow2_3mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProj
ect_Test/Content/ControlRig_Arrow2_solid.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow4_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow4_3mm.uasset'
LogSourceControl: Attempting 'p4 diff -sa U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BoundingBoxCorner.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BoundingBoxEdge.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/Cone.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow2_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow2_3mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProje
ct_Test/Content/ControlRig_Arrow2_solid.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow4_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow4_3mm.uasset'
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BoundingBoxCorner.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BoundingBoxEdge.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/Cone.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow2_1mm.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow2_3mm.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow2_solid.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow4_1mm.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow4_3mm.uasset - file(s) not opened for edit.
LogVirtualization: Display: Considering 8 file(s) for virtualization
LogVirtualization: Display: Found 8 package(s), 8 of which had payload trailers
LogVirtualization: Display: Found 8 locally stored payload(s) in 8 package(s) that maybe need to be virtualized
LogVirtualization: Display: Pushing payload(s) to cached storage is disbled, skipping
LogVirtualization: Display: Pushing payload(s) to EStorageType::Persistent storage...
LogVirtualization: [P4SourceControl - SourceControlCache] Found 8 unique payload(s)
LogSourceControl: Attempting 'p4 fstat -Or //ue_virtual/9c/3a/03/2e4000cd3e47c880eae87202a3974668e7.upayload //ue_virtual/df/58/b8/1acc44556c7fe91d1466b7655ca5429f64.upayload //ue_virtual/3a/a9/c2/1c0f050d01c05b1025101b8559ef332b5c.upayload //ue_virtual/60/08/98/e8af44e30613c480e24e3274b12087543e.upayload //ue_virtual/64/ea/5a/a790182f9375e13696281d4c9a7b728ce9.upayload //ue_virtual/0d/ff/f5/57e6cfe042a068bc0d35d845575d09fecb.upayload //ue_virtual/b3/4e/4a/a24f11e3d1aaf233c5a7fa427ae3e10e19.upayload //ue_virtual/9e/ee/97/d2cc837aa8d99242e88d960857d8bb823a.upayload'
LogVirtualization: [P4SourceControl - SourceControlCache] Determined that 8 payload(s) require submission to revision control
LogVirtualization: [P4SourceControl - SourceControlCache] Started payload submission session 'F5E7A9444BD03DBC80FAB0A09461F0D2' for '8' payload(s)
LogVirtualization: [P4SourceControl - SourceControlCache] Created directory 'U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2' to submit payloads from
SourceControl: CommandMessage Command: CreateWorkspace, Info: Client VASubmission-f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2 saved.
LogSourceControl: Switched workspaces from '' to 'VASubmission-f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2%'
LogVirtualization: [P4SourceControl - SourceControlCache] Splitting the submission into '1' batches
LogVirtualization: [P4SourceControl - SourceControlCache] Processing batch 1/1...
LogSourceControl: Attempting 'p4 add U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/9c/3a/03/2e4000cd3e47c880eae87202a3974668e7.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/df/58/b8/1acc44556c7fe91d1466b7655ca5429f64.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/3a/a9/c2/1c0f050d01c05b1025101b8559ef332b5c.upayload U:/environment_testrun/Unreal_Pr
ojects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/60/08/98/e8af44e30613c480e24e3274b12087543e.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/64/ea/5a/a790182f9375e13696281d4c9a7b728ce9.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/0d/ff/f5/57e6cfe042a068bc0d35d845575d09fecb.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-
3dbc-80fa-b0a09461f0d2/b3/4e/4a/a24f11e3d1aaf233c5a7fa427ae3e10e19.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/9e/ee/97/d2cc837aa8d99242e88d960857d8bb823a.upayload'
LogSourceControl: Attempting 'p4 reopen -c 1324 U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/9c/3a/03/2e4000cd3e47c880eae87202a3974668e7.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/df/58/b8/1acc44556c7fe91d1466b7655ca5429f64.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/3a/a9/c2/1c0f050d01c05b1025101b8559ef332b5c.upayload U:/environment_testru
n/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/60/08/98/e8af44e30613c480e24e3274b12087543e.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/64/ea/5a/a790182f9375e13696281d4c9a7b728ce9.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/0d/ff/f5/57e6cfe042a068bc0d35d845575d09fecb.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e
7a944-4bd0-3dbc-80fa-b0a09461f0d2/b3/4e/4a/a24f11e3d1aaf233c5a7fa427ae3e10e19.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2/9e/ee/97/d2cc837aa8d99242e88d960857d8bb823a.upayload'
LogSourceControl: Attempting 'p4 submit -c 1324'
SourceControl: CommandMessage Command: CheckIn, Info: Change 1324 created.
LogSourceControl: Switched workspaces from 'VASubmission-f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2' to '%'
LogSourceControl: Attempting 'p4 client -d VASubmission-f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2'
SourceControl: CommandMessage Command: DeleteWorkspace, Info: Client VASubmission-f5e7a944-4bd0-3dbc-80fa-b0a09461f0d2 deleted.
LogVirtualization: Display: Pushed 8 payload(s) to EStorageType::Persistent storage
LogVirtualization: Display: 8 package(s) had their trailer container modified and need to be updated
LogVirtualization: Display: Detaching loaded packages from disk...
LogVirtualization: Display: Reset the loaders of 8 package(s)
LogVirtualization: Display: Checking packages for write access permission...
LogVirtualization: Display: All packages have write permission
LogVirtualization: Display: Replacing old packages with the virtualized version...
LogVirtualization: Display: Total Reduction 28.708 KiB (166.469 KiB -> 137.761 KiB)
LogVirtualization: Display: Replaced 8 package(s)
LogVirtualization: Display: Virtualization process complete
LogSourceControl: Attempting 'p4 reopen -c 1325 U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BoundingBoxCorner.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/BoundingBoxEdge.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/Cone.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow2_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow2_3mm.uasset U:/environment_testrun/Unreal_Projects/Environments/Ne
wProject_Test/Content/ControlRig_Arrow2_solid.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow4_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow4_3mm.uasset'
LogSourceControl: Attempting 'p4 submit -c 1325'
SourceControl: CommandMessage Command: CheckIn, Info: Change 1325 created.
SourceControl: Submitted changelist 1325
LogAutomationController: Ignoring very large delta of 4.64 seconds in calls to FAutomationControllerManager::Tick() and not penalizing unresponsive tests
LogSlate: Window 'Submit Files' being destroyed
LogSourceControl: Attempting 'p4 fstat -Or U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow_3mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow_solid.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Box_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Box_3mm.uasset U:/environment_testrun/Unreal_Projects/
Environments/NewProject_Test/Content/ControlRig_Box_solid.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Circle_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Circle_3mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Circle_solid.uasset'
LogSourceControl: Attempting 'p4 diff -sa U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow_3mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow_solid.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Box_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Box_3mm.uasset U:/environment_testrun/Unreal_Projects/E
nvironments/NewProject_Test/Content/ControlRig_Box_solid.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Circle_1mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Circle_3mm.uasset U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Circle_solid.uasset'
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow_1mm.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow_3mm.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Arrow_solid.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Box_1mm.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Box_3mm.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Box_solid.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Circle_1mm.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Circle_3mm.uasset - file(s) not opened for edit.
SourceControl: CommandMessage Command: UpdateStatus, Info: U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Content/ControlRig_Circle_solid.uasset - file(s) not opened for edit.
LogVirtualization: Display: Considering 9 file(s) for virtualization
LogVirtualization: Display: Found 9 package(s), 9 of which had payload trailers
LogVirtualization: Display: Found 9 locally stored payload(s) in 9 package(s) that maybe need to be virtualized
LogVirtualization: Display: Pushing payload(s) to cached storage is disbled, skipping
LogVirtualization: Display: Pushing payload(s) to EStorageType::Persistent storage...
LogVirtualization: [P4SourceControl - SourceControlCache] Found 9 unique payload(s)
LogSourceControl: Attempting 'p4 fstat -Or //ue_virtual/a7/00/22/6ee0e37f8d4058ad8c953397b13919bfd8.upayload //ue_virtual/b4/a3/55/e47f72dd5ebbb493951f2754e203f1da6e.upayload //ue_virtual/53/bc/5c/946f4cb79acdb55f08366106b8c63eae60.upayload //ue_virtual/97/98/f2/3ccff21d767736f2915bbe27559be7c0fb.upayload //ue_virtual/27/50/84/71e54238247796a5c3ce68135947a21843.upayload //ue_virtual/cf/a7/42/df2d6393c8fd6e01a74883955d59349e5c.upayload //ue_virtual/f9/9a/58/84a5e8775d89fa5880776d3f9e6de05b43.upayload //ue_virtual/81/ac/b2/c5080a5b0d8b0ef4c177c22c80722a4dc9.upayload //ue_virtual/5d/34/e8/8469afd
73d805ae1136ab53a32c6e9a4a3.upayload'
LogVirtualization: [P4SourceControl - SourceControlCache] Determined that 9 payload(s) require submission to revision control
LogVirtualization: [P4SourceControl - SourceControlCache] Started payload submission session '56417E22495004981E171DAF338D05B3' for '9' payload(s)
LogVirtualization: [P4SourceControl - SourceControlCache] Created directory 'U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3' to submit payloads from
SourceControl: CommandMessage Command: CreateWorkspace, Info: Client VASubmission-56417e22-4950-0498-1e17-1daf338d05b3 saved.
LogSourceControl: Switched workspaces from '' to 'VASubmission-56417e22-4950-0498-1e17-1daf338d05b3%'
LogVirtualization: [P4SourceControl - SourceControlCache] Splitting the submission into '1' batches
LogVirtualization: [P4SourceControl - SourceControlCache] Processing batch 1/1...
LogSourceControl: Attempting 'p4 add U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/a7/00/22/6ee0e37f8d4058ad8c953397b13919bfd8.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/b4/a3/55/e47f72dd5ebbb493951f2754e203f1da6e.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/53/bc/5c/946f4cb79acdb55f08366106b8c63eae60.upayload U:/environment_testrun/Unreal_Pr
ojects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/97/98/f2/3ccff21d767736f2915bbe27559be7c0fb.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/27/50/84/71e54238247796a5c3ce68135947a21843.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/cf/a7/42/df2d6393c8fd6e01a74883955d59349e5c.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-
0498-1e17-1daf338d05b3/f9/9a/58/84a5e8775d89fa5880776d3f9e6de05b43.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/81/ac/b2/c5080a5b0d8b0ef4c177c22c80722a4dc9.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/5d/34/e8/8469afd73d805ae1136ab53a32c6e9a4a3.upayload'
LogSourceControl: Attempting 'p4 reopen -c 1326 U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/a7/00/22/6ee0e37f8d4058ad8c953397b13919bfd8.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/b4/a3/55/e47f72dd5ebbb493951f2754e203f1da6e.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/53/bc/5c/946f4cb79acdb55f08366106b8c63eae60.upayload U:/environment_testru
n/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/97/98/f2/3ccff21d767736f2915bbe27559be7c0fb.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/27/50/84/71e54238247796a5c3ce68135947a21843.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/cf/a7/42/df2d6393c8fd6e01a74883955d59349e5c.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/564
17e22-4950-0498-1e17-1daf338d05b3/f9/9a/58/84a5e8775d89fa5880776d3f9e6de05b43.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/81/ac/b2/c5080a5b0d8b0ef4c177c22c80722a4dc9.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/5d/34/e8/8469afd73d805ae1136ab53a32c6e9a4a3.upayload'
LogSourceControl: Attempting 'p4 submit -c 1326'
LogSourceControl: P4 execution time: 0.2467 seconds. Command: submit -c 1326
LogSourceControl: Attempting 'p4 reopen -c default U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/a7/00/22/6ee0e37f8d4058ad8c953397b13919bfd8.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/b4/a3/55/e47f72dd5ebbb493951f2754e203f1da6e.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/53/bc/5c/946f4cb79acdb55f08366106b8c63eae60.upayload U:/environment_tes
trun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/97/98/f2/3ccff21d767736f2915bbe27559be7c0fb.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/27/50/84/71e54238247796a5c3ce68135947a21843.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/cf/a7/42/df2d6393c8fd6e01a74883955d59349e5c.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/
56417e22-4950-0498-1e17-1daf338d05b3/f9/9a/58/84a5e8775d89fa5880776d3f9e6de05b43.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/81/ac/b2/c5080a5b0d8b0ef4c177c22c80722a4dc9.upayload U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission/56417e22-4950-0498-1e17-1daf338d05b3/5d/34/e8/8469afd73d805ae1136ab53a32c6e9a4a3.upayload'
LogSourceControl: Attempting 'p4 change -d 1326'
SourceControl: Error: CommandMessage Command: CheckIn, Error: Transfer of librarian file '//ue_virtual/27/50/84/71e54238247796a5c3ce68135947a21843.upayload' failed.
SourceControl: Error: CommandMessage Command: CheckIn, Error: Transfer of librarian file '//ue_virtual/53/bc/5c/946f4cb79acdb55f08366106b8c63eae60.upayload' failed.
SourceControl: Error: CommandMessage Command: CheckIn, Error: Transfer of librarian file '//ue_virtual/5d/34/e8/8469afd73d805ae1136ab53a32c6e9a4a3.upayload' failed.
SourceControl: Error: CommandMessage Command: CheckIn, Error: Transfer of librarian file '//ue_virtual/81/ac/b2/c5080a5b0d8b0ef4c177c22c80722a4dc9.upayload' failed.
SourceControl: Error: CommandMessage Command: CheckIn, Error: Transfer of librarian file '//ue_virtual/97/98/f2/3ccff21d767736f2915bbe27559be7c0fb.upayload' failed.
SourceControl: Error: CommandMessage Command: CheckIn, Error: Transfer of librarian file '//ue_virtual/a7/00/22/6ee0e37f8d4058ad8c953397b13919bfd8.upayload' failed.
SourceControl: Error: CommandMessage Command: CheckIn, Error: Transfer of librarian file '//ue_virtual/b4/a3/55/e47f72dd5ebbb493951f2754e203f1da6e.upayload' failed.
SourceControl: Error: CommandMessage Command: CheckIn, Error: Transfer of librarian file '//ue_virtual/cf/a7/42/df2d6393c8fd6e01a74883955d59349e5c.upayload' failed.
SourceControl: Error: CommandMessage Command: CheckIn, Error: Transfer of librarian file '//ue_virtual/f9/9a/58/84a5e8775d89fa5880776d3f9e6de05b43.upayload' failed.
SourceControl: Error: CommandMessage Command: CheckIn, Error: Submit aborted -- fix problems then use 'p4 submit -c 1326'.
SourceControl: CommandMessage Command: CheckIn, Info: Change 1326 created.
SourceControl: CommandMessage Command: CheckIn, Info: Change 1326 deleted.
LogVirtualization: Error: [P4SourceControl - SourceControlCache] Failed to submit the payload file(s) to revision control
LogSourceControl: Switched workspaces from 'VASubmission-56417e22-4950-0498-1e17-1daf338d05b3' to '%'
LogSourceControl: Attempting 'p4 client -d VASubmission-56417e22-4950-0498-1e17-1daf338d05b3'
SourceControl: Error: CommandMessage Command: DeleteWorkspace, Error: Client 'VASubmission-56417e22-4950-0498-1e17-1daf338d05b3' has files opened. To delete the client, revert any opened files and delete any pending changes first. An administrator may specify -f to force the delete of another user's client.
LogVirtualization: Warning: [P4SourceControl - SourceControlCache] Failed to remove temp workspace 'VASubmission-56417e22-4950-0498-1e17-1daf338d05b3' please delete manually
LogVirtualization: Error: [P4SourceControl - SourceControlCache] Failed to push '9' payload(s)
LogVirtualization: Error: Failed to push '9' payload(s) to any backend'
LogVirtualization: Display: Virtualization process complete
SourceControl: Error: Failed to push payloads

Paul · January 16, 2025, 8:02am

Hello,

If you submit 9 asset’s in a single changelog with p4 backend, it will always fail, as it deletes the asset’s AFTER they are “added” but BEFORE they are submitted, resulting in a librarian error.

Did you confirm this by stepping through the submit in the debugger and you saw the files being removed from windows explorer between the FMarkForAdd command and the FCheckIn command or are you making an assumption based on the perforce error message? We do run a clean up pass when the p4 submit command fails so it is possible that the submit is failing for other reasons.

If you are sure that the files are being deleted at this point then my first suspicion would be interference from your anti virus software although why this would only occur with more than eight files I don’t know. You could try running Process Monitor from SystemInternals during a submit and filter for file events in your U:/environment_testrun/Unreal_Projects/Environments/NewProject_Test/Saved/VASubmission directory to see what is actually deleting the files.

You did also mention that changing the batch size has no effect. So if you set the batch size to 4 and try to submit 10 or so assets you were still seeing the same Transfer of librarian file errors even though we’d only be submitting 4 files at a time? If so, is it the first batch that is failing or one of the subsequent ones?

Let me know what you find out as it will help narrow down the problem on our end.

This isn’t related to your problem but from the log you posted I did notice that with the successful submit you were virtualizing really small data sets

LogVirtualization: Display: Total Reduction 162.165 KiB (267.243 KiB → 105.078 KiB)

which probably isn’t worth the overhead for the minor disk space saving. Consider setting the MinPayloadLength to something higher (we’ve found 128KiB to 512 Kib is a good value range depending on your data set) to cut down on data churn. See the code doc in Engine\Source\Developer\Virtualization\Private\VirtualizationManager.h

TristonS · January 17, 2025, 10:49am

Hey Paul, I ended up resolving this, a bit of a facepalm moment.

We run P4 locally, Parallell sync/pull is supported, but parallell submission is not on Windows.

https://help.perforce.com/helix-core/server-apps/cmdref/current/Content/CmdRef/configurables.alphabetical.html#net.parallel.shelve.threads

When reviewing the documentation, I noticed a random config value at 9. Unfortunately, they don’t mention this limitation on the parallell documentation page, and it is only briefly mentioned as a fotonote on the p4 submit docs page.

I was not aware my local server would be considered an edge server in this case, but just disabling parallell submissions worked. We can still do threaded sync’s, and the virtual asset’s work fantasticaly. Even with large latency, the whole process has been so smooth since. Thanks for your efforts!

So, tldr, p4 bug. I think perhaps the output could be a little more detailed though to indicate if a failure is occuring and it’s not because of the virtual system or as a footnote in the docs for those on Windows.

https://help.perforce.com/helix-core/server-apps/cmdref/current/Content/CmdRef/p4_submit.html

Important

Limitation: Parallel submit is not supported from an edge server if the commit server is on a Windows platform. In this case --parallel or automatic parallel processing for submit will be silently ignored, and the submit will run without using parallel threads.

Paul · January 17, 2025, 1:05pm

Thanks for the detailed reply!

It should be enough to attempt to build a repro case on our end although I cannot make any promises that I will be able to get time to follow up on this as it doesn’t sound like there would be much that we could about it.

However I do agree that in this case the logging could’ve been better, so I’ve added a work item to do a pass on that. This is something that I am much more likely to be able to get to in the near future.