In part one of this series, we discussed how success notifications from your backup tool of choice can create a false sense of confidence that backup restores will work when needed. We also took a look at SureBackup, a component of the Enterprise and Enterprise Plus editions of Veeam Backup and Replication that performs automated restore testing. As promised, we’re going to dive into each of the SureBackup components. Go grab a cup of coffee and get comfortable, as there’s a lot of information that follows….
A virtual lab, created by Veeam Backup, is normally a tool an IT admin would use to perform testing in an isolated environment. For example, a team of developers could spin up a group of SQL virtual machines (VMs) and test code changes without affecting production VMs. Veeam takes this technology further to spin up VMs from a restore to see if they actually work.
Here’s how to set-up a virtual lab in Veeam Backup 7.0 for VMware. Note: the following steps have been pulled directly from Veeam’s support site, sans screenshots.
To get started…
Open the Backup Infrastructure view, select the Virtual Labs node under SureBackup in the inventory pane and click Add Virtual Lab > VMware on the ribbon.
Open the Backup Infrastructure view, right-click the Virtual Labs node under SureBackup in the inventory pane and select Add Virtual Lab > VMware.
At the Name step of the wizard, enter a name and description for the new virtual lab. The default description contains information about the user who created the lab, as well as the date and time when the lab was created.
Selecting an ESX(i) Host for VM Backups Verification
To select an ESX(i) host for VM backups recovery verification:
You cannot create resource pools in clusters with disabled DRS. If the destination host is a part of such a cluster, the Create a designated resource pool option will be disabled in the Destination Options window. See more details from VMware.
You cannot create folders on standalone ESX(i) hosts or ESX(i) hosts that are managed by the vCenter Severs but are added to Veeam Backup & Replication console as standalone hosts. To overcome this situation, add the corresponding vCenter Server to the Veeam Backup & Replication console.
Click Choose to select a datastore on which redo logs for verified VMs should be placed. Redo logs are auxiliary files used to store all changes that take place when a VM runs from a read-only backup. As soon as the recovery verification job completes, redo logs are deleted.
At this step of the wizard, you should configure proxy appliance settings.
If you assign to the proxy appliance an IP address from the same network where the Veeam backup server is located, Veeam Backup & Replication will automatically add a new route to the routing table on the Veeam backup server. If you assign to the proxy appliance an IP address from a different network, you will have to manually add a new route to the routing table on the router in the production network. If you do not add a new route, tests and application scripts will fail and you will not be able to access VMs in isolated networks.
Select the type of network settings configuration. This step differs for virtual labs used for VM backups and VM replicas verification.
Selecting a Networking Mode for VM Backups Verification
Veeam Backup & Replication offers two networking modes for the virtual lab in which VMs from backups are verified:
Basic single-host (automatic configuration). This networking mode is recommended if all VMs you plan to verify, VMs from the application group and Veeam backup server are located in the same production network. In this case, Veeam Backup & Replication will automatically define all networking settings and create the virtual lab on an ESX(i) host.
Advanced single-host. This networking mode is recommended if VMs you plan to verify and/or VMs from the application group are located in different networks. In this case, you will have to manually define settings for isolated networks in the virtual lab. Veeam Backup & Replication will use the specified parameters to create the virtual lab on an ESX(i) host.
If this option is selected, the New Virtual Lab wizard will include additional steps for customizing network settings.
This step is available if you have selected the Advanced networking option at the Networking step of the wizard.
At this step of the wizard, you should create isolated networks to which verified VMs and VMs from the application group will be connected and map these networks to production networks where initial VMs are located.
To add a network:
You can map several production networks to the same isolated network. The production networks you plan to map to the same isolated networks must have the same network masks and pools of IP addresses for mapping to be configured correctly.
At this step of the wizard, you should specify settings for every created isolated network and define how production networks should map to networks in the isolated virtual lab.
Communication between production networks and isolated networks is carried out through vNIC adapters that are added to the proxy appliance. A new vNIC adapter is added for every isolated network.
To add a vNIC adapter for an isolated network:
Network addresses for different network adapters should belong to different networks. For example, if the first network adapter has address 192.168.0.1 with mask 255.255.255.0, and the second one — 192.168.0.2 with mask 255.255.255.0, such configuration will not work. In this situation, you need to assign to the second adapter the IP address from a different network, for example, 172.16.0.1.
This step is available if you have selected the advanced networking option at the Networking step of the wizard.
At this step of the wizard, you can specify static IP address mapping rules to make VMs in the virtual lab accessible from any machine in the production network.
To add a new rule:
For example, the DNS server you plan to start in the virtual lab has IP address 192.168.1.2 in the production network. To set static mapping for the DNS server, in the Isolated IP field, you need to define its production IP address — 192.168.1.2. In the Access IP field, you need to define any unallocated IP address from the production network, for example, 192.168.1.48. After a virtual lab is created and VMs are started in the virtual lab, you will be able to access the DNS server in the virtual lab from the production environment using IP address 192.168.1.48.
Review the parameters of the virtual lab you create. You can go back to any previous step to adjust the parameters. If everything is fine, click Next to create the virtual lab.
Always use Veeam Backup & Replication to modify or delete a virtual lab. If you change lab settings or delete any of its components from outside (for example, using vSphere Client), the lab will be corrupted and its component such as the created vSwitch, resource pool and so others will remain in the virtual infrastructure.
So, that’s it for setting up the Virtual Lab.
“Um…I thought you said that Veeam would do this for me automagically? What’s with all the work?” Well, there are a few reasons for that. First, Veeam doesn’t “know” what your virtual infrastructure looks like, so the above steps help with “describing” the environment to Veeam. Secondly, as mentioned before, a virtual lab can be used for other purposes besides validating a restore.
Having granular control over what a virtual lab consists of provides flexibility and added functionality. On to the next component of the SureBackup toolset:
Ever booted up an Exchange server without a domain controller present? It will stare at you as though you’ve lost your mind. Veeam understands that a valid restore test requires that dependent VMs be running, so they allow for the creation of an application group. An application group is a group of VMs that have dependencies on each other. For example, you might want to test a restore of an IIS server that’s providing web services tied to a backend SQL server. VM dependencies are important to understand, because you can’t test functionality of certain VMs without that understanding. Let’s return to Veeam’s support site to see how to build out an application group.
Getting started:
To create a new application group, do either of the following:
Open the Backup Infrastructure view, select the Application Groups node under SureBackup in the inventory pane and click Add App Group > VMware on the ribbon.
Open the Backup Infrastructure view, right-click the Application Groups node under SureBackup in the inventory pane and select Add Application Group > VMware.
At the Name step of the wizard, enter a name and description for the new application group. The default description contains information about the user who created the group, as well as the date and time when the group was created.
At the Virtual Machines step of the wizard, you should add VMs to the created application group. You can add VMs from VM backups or VM replicas.
Veeam Backup & Replication supports mixed application groups. You can add to the same application groups VMs from backups and VMs from replicas. Keep in mind that all VMs from the application group must belong to the same platform — VMware or Hyper-V, and must have at least one valid restore point created by the time the SureBackup job starts.
Adding VMs from VM Backups
To add a VM from the backup to the group, click Add Backup and select where to browse for VMs:
From Infrastructure to browse the virtual environment. Because VMs from the application group are started from VM backups, you need to make sure that VMs you have selected for the group have been successfully backed up at least once by the time you plan to run a SureBackup job.
From Backups to browse existing backups.
After you have added necessary VMs to the application group, you should specify a role, VM startup options and select tests to be performed for each VM in the application group.
To be able to perform tests, Veeam Backup & Replication requires VMware Tools to be installed on the verified VM. If VMware Tools are not installed, the VM will be started but tests will not be performed.
VMs without VMware Tools can still be used as auxiliary VMs that should be started to enable proper work of verified VMs or VM replicas. In this case, you do not need to select any role for such auxiliary VM.
Select the necessary VM in the list and click Edit on the right.
Role settings
On the Role tab, select the role that a VM performs. Veeam Backup & Replication offers the following predefined roles for VMs:
VM roles are described in XML files stored in the %Program Files%\Veeam\Backup and Replication\Backup\SbRoles folder. You can add your own roles. To do this, you need to create a new XML file and specify role and test scripts settings in it. For details, see Creating XML files with VM Roles Description.
Once you select the necessary role, Veeam Backup & Replication will automatically configure startup options and provide predefined test scripts applicable for the chosen role. You can use these settings or specify custom ones using the Startup Options and Test Scripts tabs.
To verify VMs that perform roles other than those specified in the list, you will have to manually configure startup options and specify test scripts to be run against these VMs.
Startup Options
On the Startup Options tab, specify VM startup setting.
Before you start a SureBackup job, make sure that the firewall on the tested VM allows ping requests.
Be careful when specifying the Maximum allowed boot time value. Typically, a VM started by a SureBackup job requires more time to boot than a VM started regularly. If an application fails to be initialized within the specified interval of time, the recovery verification process will be finished with the timeout error. If such error situation occurs, you will need to increase the Maximum allowed boot time value and start the job again.
Test Scripts
On the Test Scripts tab, specify what test scripts should be run to verify a VM. When you select a VM role, Veeam Backup & Replication automatically assigns a predefined script that will be run to verify applications inside the VM.
If you want to verify a VM that has some other role not listed on the Role tab, do the following:
To edit test settings, select the test in the list and click Edit. To delete a test, select it in the list and click Remove.
If a VM performs several roles running a number of applications at once, you can verify their work by adding several verification scripts. For such VMs, it is recommended to specify the maximum startup timeout value and allocate the greatest amount of memory.
After the group is created, review the application group summary and click Finish to exit the wizard.
Okay, troops – we’re almost home. Now that we have a Virtual Lab and an Application Group, we can move on to creating the SureBackup job.
SureBackup
This is where the “automagically” part of this whole thing actually happens. Here are instructions for:
To create a new SureBackup job, do either of the following:
Enter a name and description for the new SureBackup job. The default description contains the time at which the job was created and user who created it.
From the Virtual lab list, select one of existing virtual labs in which recovery verification should be performed. The list contains all virtual labs that were created or connected to the Veeam backup server. Information about the selected virtual lab is displayed in the Virtual lab info section.
At this step of the wizard, you should define an application group that should be used to enable full functionality of verified VMs.
You can select an application group for recovery verification or skip this step. If the application group is not selected, you must link a backup job to the created SureBackup job at the next step. In this case, when the SureBackup job is started, Veeam Backup & Replication will only run VMs from the linked backup job in the virtual lab and verify them.
To select an application group:
At this step of the wizard, you should select VM backups or replicas that you want to verify with the created SureBackup job. Once you run a SureBackup job, Veeam Backup & Replication will start VMs from the application group in the required order and then boot and verify VMs from the linked backups or replicas.
You can link a backup or replica to the SureBackup job or skip this step. If you do not link a backup or replica to the SureBackup job, Veeam Backup & Replication will only start VMs from the application group in the virtual lab and verify them. You have an option not to link a backup or replica to the SureBackup job only in case you have selected an application group at the previous step of the wizard.
To link a backup or replica job to the SureBackup job:
To remove a backup or replica job from the list, select it and click Remove.
After you link a backup job with VMs you want to verify, you should define roles, specify startup options and select tests to be performed for these VMs.
If all VMs in the linked backup(s) perform the same role, you can specify startup options and test settings for the whole backup job in bulk. If VMs have different roles, you can granularly set startup options and select tests to be used for each VM in the backup job.
If you specify startup options and tests individually for a VM, Veeam Backup & Replication will apply these options and tests only. Options and tests specified at the level of the SureBackup job will be ignored for this VM.
To be able to perform tests, Veeam Backup & Replication requires VMware Tools to be installed on the verified VM. If VMware Tools are not installed, the VM will be started but tests will not be performed. VMs without VMware Tools can still be used as auxiliary VMs that should be started to enable proper work of other VMs. In this case, you do not need to select any role for the VM.
Role settings
On the Role tab, select the role that a VM performs. Veeam Backup & Replication offers the following predefined roles for VMs:
VM roles are described in XML files stored in the %Program Files%\Veeam\Backup and Replication\Backup\SbRoles folder. You can add your own roles. To do this, you need to create a new XML file and specify role and test scripts settings in it. For details, see Creating XML files with VM Roles Description.
Once you select the necessary role, Veeam Backup & Replication will automatically configure startup options and provide predefined test scripts applicable for the chosen role. You can use these settings or specify custom ones using the Startup Options and Test Scripts tabs.
To verify VMs that perform roles other than those specified in the list, you will have to manually configure startup options and specify test scripts to be run against these VMs.
Startup Options
On the Startup Options tab, specify VM startup settings:
Before you start a SureBackup job, make sure that the firewall on the tested VM allows ping requests.
Be careful when specifying the Maximum allowed boot time value. Typically, a VM started by a SureBackup job requires more time to boot than a VM started regularly. If an application fails to be initialized within the specified interval of time, the recovery verification process will be finished with the timeout error. If such error situation occurs, you will need to increase the Maximum allowed boot time value and start the job again.
Test Scripts
On the Test Scripts tab, specify what test scripts should be run to verify a VM. When you select a VM role, Veeam Backup & Replication automatically assigns a predefined script that will be run to verify applications inside the VM.
If you want to verify a VM that has some other role not listed on the Role tab, do the following:
To edit test settings, select the test in the list and click Edit. To delete a test, select it in the list and click Remove.
If a VM performs several roles running a number of applications at once, you can verify their work by adding several verification scripts. For such VMs, it is recommended to specify the maximum startup timeout value and allocate the greatest amount of memory.
On the Settings step, you can specify additional settings for the SureBackup job:
If you select the Keep the application group running once the job completes option at the Application Group step of the wizard, the Skip validation for application groupoption will be automatically enabled.
At the Schedule step of the wizard, you can select to manually run the SureBackup job or schedule the job at specific time, for example, after the corresponding backup job completes.
Review the summary of the created recovery verification job. Select the Run the job when I click Finish check box to start the created job right after you finish working with the wizard; then click Finish.
That was indeed a lot of material! But here’s the good news: It will take most IT admins less than an hour to step through this whole guide, and that’s well worth the peace of mind of knowing that your backups are working, not worthless.
“I have the Standard version of Veeam Backup and Replication. What do I do?” Naturally, you could consider upgrading. But if you aren’t ready for that just yet, keep an eye out for part three of this series where we will discuss how to perform manual restore testing.