Traditional agent-based antivirus does not play well in virtualized environments, whether it be server or desktop. When resources are consolidated, there can be major performance impacts during scans, including disk IO and CPU overhead. You might also notice poor application performance or slow logon times – which leads to poor user experience — if these things are happening. I’ve had a SAN crash every day due to an antivirus storm, which caused major downtime. It’s important that you carefully consider the ramifications your security software can have on your applications and infrastructure when you set it up.
The way traditional antivirus works is that you install a software agent on each computer and each agent will download definitions on a regularly scheduled basis, as well as perform on-demand scans and/or real-time scanning operations. These operations can be very taxing on system performance, such as CPU and disk IO, which causes latency. Taking this to a virtual environment where you have the potential of hundreds of virtual machines sharing the same infrastructure, you can see massive performance impact on back-end resources during scanning or update operations. We usually call this an antivirus storm. If your storage is not spec’d to handle this additional IO, you can potentially experience a severe performance impact.
To combat this, you can either accommodate with high-performing storage, which can be costly, or you use antivirus offloading technology like NSX Guest Introspection (formerly known as vShield Endpoint) in conjunction with your preferred antivirus vendor, like Deep Security from Trend Micro. Side note: you can get this feature in VMware without having to shell out for the full NSX license. It comes with vSphere Essentials Plus or higher.
What guest introspection does is it takes the scanning operations off each virtual machine, eliminating the need for an agent on each one, and offloading it to a service virtual machine that resides on each host. That allows the scanning and updates to be done at the hypervisor level instead of the guest OS. The main advantage of this is to reduce the strain on CPU and disk resources. So now, instead of hundreds of VMs running the updates/scanning at the same time, you only have one service VM per host that handles the antivirus/antimalware operations for all VMs residing on that host.
I Already Have Antivirus Agents On All My VMs and Switching Is Not An Option: What Do I Do?
If you’ve invested in antivirus already and you don’t have the option to upgrade to something like Trend Micro Deep Security and NSX guest introspection, then you need to make sure you do your best to follow your software vendors’ recommendations. Every application that’s installed on your virtual server or desktops has best practices in regards to antivirus policies. (Regardless of whether or not you’re offloading antivirus operations, this should be done whenever deploying antivirus.) For example, here’s Microsoft’s exclusion list for Windows. Here’s VMware’s exclusion list for Horizon 7. These lists are not short.
For remote desktop services or Windows desktops that use roaming profiles and/or folder redirection, you will need to make exclusions in your antivirus policy for the UNC paths to all scanning operations and any mapped network drives. The reason for this is to allow the file server to do the heavy lifting by scanning the files going in and out of the server. This, essentially, can reduce a lot of the workload on the desktops and allows the file server to do the work on their behalf, doing it just once instead of on all machines. This will prevent bogging down logon times by eliminating the scanning on the files copying down on the desktops at the time of logon. This also applies to user environment virtualization solutions like VMware UEM, Ivanti (AppSense) Environment Manager or VMware personas, etc. If you’re using any of these solutions, refer to that vendor’s AV guides to ensure that you’ve set everything up according to best practice recommendations.
Again, it’s very important to understand how your applications operate in order to fine tune your exclusion list and your behavior monitoring.
Additionally, randomizing scheduled scans can reduce the overhead during a scanning operation. Instead of scanning all desktops or all servers at the same time, you can schedule them in smaller groups and randomize the scanning schedules so they won’t compete for resources at the same time.
Antivirus is a necessary evil and it can create a world of hurt if you don’t take the time to understand how it’s set up and works within your environment. It keeps things secure and you need it, but it can create tremendous performance problems. (This can be true any time you push out default policies without consideration for anything.) And when you add resource-hungry antivirus operations into a virtual environment, which requires the sharing resources, you can compound those performance strains greatly.
The whole idea here is for your antivirus to keep you secure, but to run as efficiently as possible, so whitelist all you can and follow all your vendors’ recommendations.