Antivirus and Large-Scale Asset Processing

Development Asset Creation
, ,
1

Hello,

I’m mainly looking for feedback from others who may have faced a similar issue.

In our pipeline, before packaging, we duplicate levels in order to perform a number of optimization steps, such as merging dynamic meshes and converting them into static meshes. Our levels currently contain between 100,000 and 150,000 actors.

The issue we’ve identified is that processing times can increase by a factor of 4 to 5 when an antivirus is running (we are currently using Symantec). Basically, in my case, my line compilation takes more than 20 hours instead of 5 hours . We’ve tried excluding the directories where these operations take place, and we’ve also tried disabling the antivirus (without uninstalling it), but processing times remain significantly longer.

As we understand it, Symantec consists of both a daemon and a kernel driver that intercepts a number of system calls, particularly those related to on-access file scanning. It intercepts file open, read, and write operations and scans files before allowing them to proceed.

For now, we’ve uninstalled the antivirus on a few isolated machines that are disconnected from the network and use those machines for our build process. However, this is not a viable long-term solution. Has anyone encountered similar issues? If so, how did you address them?

Thanks,

Philippe

[Attachment Removed]

2

Hi,

Several years ago, in a past job, we had similar issues and we were able to exclude certain folders from the anti-virus (Carbon Black), but it looks like you already tried that with Symantec. According to Google though, it should be possible to exclude folders from Symantec Endpoint Protection (SEP) kernel-level or real-time scanning. You need to configure an Exceptions Policy in the management console:

But I’m assuming you already did that and you excluded all the folders used by the process, you might want to use another anti-virus that support ignoring particular folders or add some automation to turn the Symantec SEP on/off in between work.

Regards,

Patrick

[Attachment Removed]

3

Thank you, Patrick, for taking the time to answer.

We did try excluding several directories from the antivirus scan, but for some reason the file system hook is still active. What I find strange is that it still spends a significant amount of time processing files that should have been excluded.

In any case, I’ll discuss this with our IT team and see whether we can at least test a different antivirus solution.

Thank you again,

Philippe

[Attachment Removed]