Configure Direct NFS Mode in Veeam Backup and Replication 9

Veeam introduced in Backup and Replication version 9 a new transport mode called Direct NFS. It’s similar to Direct SAN transport mode where you present the FC or iSCSI LUNs to the Backup Proxy Server. In the case of Direct NFS you allow access to the NFS datastore from every Veeam Backup Proxy which uses the Direct NFS mode. Direct NFS is written from scratch by Veeam and doesn’t leverage the VDDK for backup and restore.

History

If you used Veeam Backup and Replication prior to v9 then the following transport modes are only available for backing up VMs on NFS datastores:

• Virtual Appliance Mode aka Hot-Add Mode
• Network Transport Mode aka NBD Mode

Virtual Appliance Mode was my preferred mode when doing backups in my home lab but I also saw that some of the VMs, most of the time vCenter and Domain Controller get stunned during the backup operations. The reason for this is that a NFS datastore doesn’t have a native capability like VMFS to determine which ESXi host is the owner of a particular VM. NFS datastores relies on a LCK file (e.g. DC01.vmx.lck) that resides within the VM folder. During hot-add operations, the host on which the Hot-Add proxy resides will temporary take ownership of the VM by changing the contents of the LCK file. If the VM being backed up is on a different ESXi host then the Hot-Add proxy this could lead to either:

• additionals “stuns” of the VM
• VM become unresponsive for approximately 30 seconds
• or the snapshot removal takes a long time to complete

This is unfortunately an issue with the NFSv3 protocol and the locking method. If your using NFSv4 datastores (vSphere 6.x or higher) you shouldn’t have this problem. For more information see VMware KB2010953.
You can also mitigate this issue by deploying a Hot-Add proxy to every ESXi host where VMs reside that you want to backup by this method. In addition you also need to create the following Registry key on the Veeam Backup and Replication Server:

• Path: HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication\
• DWORD Key: EnableSameHostHotAddMode
• Value: Default Value is 0
  • value: 1 (if proxy on same host as VM is unavailable, Veeam Backup & Replication will fail over to a proxy on a different host and use available transport mode, which may cause stun)
  • value: 2 (if proxy on same host as VM is unavailable, Veeam Backup & Replication will use an available proxy on a different host, but force it to use network transport mode, so that no stun occurs; this may be preferable when stun is not tolerable)

If you can’t deploy backup proxies to every ESXi host you should use Network Transport Mode as transport mode instead.
Network Transport Mode is the slowest mode of the VDDK transport modes. It leverages the VMware Network File Copy (NFC) protocol which is limited for a maximum throughput of 40% of the VMkernel interface (1Gbit => approx. 51,2 MB/s or 409,6 Mbit, 10Gbit => approx. 512MB/s or 4096 Mbit). The NBD mode traffic always goes through the VMkernel which is selected as Management Interface. So if you plan to use NBD, design your solution with this “limitation” in mind.

Infrastructure

The infrastructure I’m using for Direct NFS mode consists of the following:

• Synology DS1515+, 4x WD Red 3TB, 2x 1Gbit for Management (Bond ALB), 2x 1Gbit for iSCSI and NFS (Bond ALB)
• ESXi 5.5 U3 host, 80GB Memory, 4x 1Gbit Network interfaces
• VBR-Server, Windows 2012 R2, 4vCPU, 6GB Memory
• VBR-Proxy, Windows 2012 R2, 4vCPU, 4GB Memory
• IP NFS Storage 172.16.90.204, IP NFS NIC Proxy 172.16.90.16
• 2x NFS datastores (NFS01, NFS02) both connected to the ESXi host

Direct NFS Requirements

• The backup proxy used for VM data processing must have access to the NFS datastore(s) where VM disks are located
• The backup proxy must have ReadOnly/Write permissions and root access to the NFS datastore.
• If NFS volumes are mounted on the ESX(i) host under names, not IP addresses, the volume names must be resolved by DNS from the backup proxy

Direct NFS limitations

• Veeam Backup & Replication cannot parse delta disks in the Direct NFS access mode. For this reason, the Direct NFS access mode has the following limitations:
  • The Direct NFS access mode cannot be used for VMs that have at least one snapshot.
  • Veeam Backup & Replication uses the Direct NFS transport mode to read and write VM data only during the first session of the replication job. During subsequent replication job sessions, the VM replica will already have one or more snapshots. For this reason, Veeam Backup & Replication will use another transport mode to write VM data to the datastore on the target side. The source side proxy will keep reading VM data from the source datastore in the Direct NFS transport mode.
• If you enable the Enable VMware tools quiescence option in the job settings, Veeam Backup & Replication will not use the Direct NFS transport mode to process running Microsoft Windows VMs that have VMware Tools installed.
• If a VM has some disks that cannot be processed in the Direct NFS access mode, Veeam Backup & Replication processes these VM disks in the Network transport mode.

Direct NFS Configuration

• Proxy server needs access to the NFS network. You can either configure routes to the NFS server or you add an additional NIC to the proxy server which is located in the NFS network. I used the second option because my NFS network doesn’t have a default gateway. This NIC is configured with the IP address 172.16.90.16.
• Configure access to the NFS datastore. In my case I added the NFS IP address of the Proxy server to my NFS share hosted on the Synology. Squash: No mapping means in case of the Synology No root squash. You can also choose to use “Read Only”  as Privilege.
  
• Configure the proxy server transport mode. Per default the transport mode is set to Automatic selection. I configured my proxy server to use only Direct storage access (required for Direct NFS) with no failback to NBD. With this config I can ensure that the backup is only working if the NFS configuration is correct. If something is not correct with the NFS config you will get this error message when you start the backup job:
  

  Unable to allocate processing resources. Error: No backup proxy is able to process this VM due to proxy processing mode restrictions.

  If you choose Automatic selection the order of the transport modes is the following:

  • Direct Storage Access
  • Virtual Appliance
  • Network
• Rescan the proxy server. During my configuration and testing my main failure was that Direct NFS was never working. I spend some days to find out what was wrong and I used a lot of tools to find the problem like Wireshark etc. If you configure the access to the NFS datastores you MUST rescan the proxy server in the Backup and Replication Console.
  Go to Backup Infrastructure – Microsoft Windows – Select your proxy server – Right click and select Rescan.
  
  Only if you do this, the proxy server will find the new access enabled NFS datastores for it.

For this article I used the following sources:

• Direct NFS Access
• Backup Proxy for Direct NFS Access Mode
• Transport Modes
• Considerations for NFS Datastores