Swarm [Pushing the channel has failed!]

Development Rendering
, ,
1

Hello,

I’m trying to bake light on multiple computers. I can see all computers and “send” them job… but.

16699-swarm.png
16699-swarm.png956×505 40.6 KB 

21:27:25: [Channel] Successful channel push of ../DotNET/AutoReporter.exe
21:27:25: [OpenChannel] Opening Channel: ../DotNET/AutoReporter.exe
21:27:25: [PushChannel] Pushing the channel has failed!
21:27:25: [PushChannel] Exception message: System.IO.DirectoryNotFoundException: Impossible de trouver une partie du chemin d'accès 'C:\Program Files\Unreal Engine\4.4\Engine\Binaries\DotNET\DotNET\AutoReporter.exe'.
   à System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   à System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   à System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   à System.IO.File.InternalReadAllBytes(String path, Boolean checkHost)
   à Agent.Agent.PushChannel(RemoteConnection Remote, String ChannelName, AgentGuid JobGuid)

I saw other post on AnswerHUB, but I didnt find real answer to that problem.

Thank you,
Sam

2

Hi Bychop,
that seems like a bad path C:\Program Files\Unreal Engine\4.4\Engine\Binaries\DotNET\DotNET\AutoReporter.exe. Could you verify if it exists on your machine?

Thanks,
Jarek

3

Replace the SwarmAgent.exe under Engine\Binaries\DotNET with this one:

[SwarmAgent.zip][1]

This problem is caused by a logical bug in Engine\Source\Programs\UnrealSwarm\Agent\Channel.cs:

For UnrealLightmass, a job may have two types of dependencies: required or optional.

24790-1.png
24790-1.png805×217 7.02 KB

The code above in Engine\Source\Programs\UnrealSwarm\Agent\Jobs.cs shows if the transferring of a required dependency fails an error will be thrown, otherwise a warning will be displayed.

24791-2.png
24791-2.png813×290 9.31 KB

But this while-loop in Engine\Source\Programs\UnrealSwarm\Agent\Channels.cs makes the check useless. bChannelTransferred will be set to false due to:

  • Network error
  • Some fatal exceptions

For situation 2, we will be stuck in this while-loop. (The exception handling code below is useless)

An ad-hoc solution is to add a variable retryCount here:

24792-3.png
24792-3.png835×311 8.85 KB

Then the problem is solved.

P.S. : You can also use the patch I mentioned here: cluster problem with swarm agent and two pc - AnswerHub - Unreal Engine Forums

4

Hi Luoshuang,
thanks for this comment! Unfortunatelly note that you are writing about “Pulling the channel has failed!” and the user asked about “Pushing the channel has failed!”. Those are two very different places in the code.

Nevertheless we are aware of the problem you’ve mentioned and it had been fixed since 4.6.

Thanks,
Jarek

5

It is Pushing the channel, because the C# method PushChannel will be called remotely by PullChannel…and PullChannel will keep trying to call PushChannel when it fails, so we’ll see a lot of errors.

Anyway, I’m happy to see it fixed in 4.6. :slight_smile:

Another problem usually happening is “ping failed”. An ad-hoc solution is just to remove the step of pinging and let the result of TryOpenConnection tell us if the remote machine is online.

Luoshuang’s SwarmAgent.zip (pinging removed)

6

I’ve checked the latest source code just now, and their technique is very similiar to mine~ :slight_smile: