Monday, July 29, 2013

Nutanix NFS VAAI troubleshooting

vStorage APIs for Array Integration is a feature introduced in ESXi that provides 
hardware acceleration functionality. It enables your host to offload specific virtual 
machine and storage management operations to compliant storage hardware. With
 the storage hardware assistance, your host performs these operations faster and 
consumes less CPU, memory, and storage fabric bandwidth.

1. Full /Fast Copy, which is used to copy or migrate data within the same physical array 
2.  RESERVE Space ( Nutanix 3.5)
3. extended statistics.

To verify if nutanix vaai is installed:

~ # esxcli software vib list |grep vaai
nfs-vaai-plugin                1.1-101                             Nutanix   CommunitySupported  2013-05-12

~ # esxcli storage  nfs list
Volume Name   Host              Share   Accessible Mounted  Read-Only  HW Acceleration
-----------------  -----------  ------------------------  -------  ---------  ---------------------
NFS-StorageA  /NFS-A   true       true           false             Supported

NFS Uses Full File Clone Primitive and Block storage uses XCOPY primitive. 
Storage vMotion, Hot Cloning cannot leverage  Full File Clone Primitive. 
Full file clone primitive can be leveraged only when a cold migration is initiated.

Reserve Space primitive is implemented  Nutanix 3.5 software version.
More details in
VMWARE kb-1021976
VAAI clone will fail:
1. if VM has snapshot
2. if vm is powered on
3. if container name has space or other characters
4. cloning between datastores.

First use ‘u’ to select device view, then enable fields o and p for VAAI stats.

VAAI Logs:

grep -i VAAI var/log/vpxa.log and vmkernel.log

Possible Errors in /var/log/vpxa.log/vmkernel.log
2012-06-21T01:30:09.165Z [25DAEB90 info 'DiskLib' opID=B2F4E13F-00001118-66]
DISKLIB-LIB   : Failed to create native clone on destination handle :
One of the parameters supplied is invalid (1).

  • Cloning to a different datastore
  • Creating a clone from a VM that has a snapshot

2012-07-18T23:15:49.378Z [4AB51B90 info 'DiskLib' opID=55E1C10D-00000188-3a]
DISKLIB-LIB   : Failed to create native clone on destination handle :
The system cannot find the file specified (25).

  • A container name that contains a space
  • A container name that was changed

2012-07-19T01:59:02.865Z [362ACB90 info 'DiskLib' opID=4A5CFD5E-00006113-5f]
DISKLIB-LIB   : Failed to create native clone on destination handle :
The specified feature is not supported by this version (24).

  • Cloning from the local datastore (VMFS)
Also you can vmware.log

2013-07-18T04:09:50.903Z| vcpu-0| SNAPSHOT: SnapshotDumperOpenFromInfo: Creating checkpoint file /vmfs/volumes/78993998-9e053604/ENCmsOLY1007-SCCM/ENCmsOLY1007-Snapshot168.vmsn
2013-07-18T04:09:51.494Z| vcpu-0| nutanix_nfs_plugin: Established VAAI session with NFS server
2013-07-18T04:09:51.526Z| vcpu-0| DISKLIB-VMFS  : "/vmfs/volumes/78993998-9e053604/ENCmsOLY1007-SCCM/ENCmsOLY1007-000002-delta.vmdk" : open successful (29) size = 1618106368, hd = 0. Type 8
2013-07-18T04:09:52.255Z| vcpu-0| DISKLIB-LIB   : DiskLibCreateCreateParam: vmfssparse grain size set to : 1