Monday, July 29, 2013

Nutanix NFS VAAI troubleshooting

vStorage APIs for Array Integration is a feature introduced in ESXi that provides 
hardware acceleration functionality. It enables your host to offload specific virtual 
machine and storage management operations to compliant storage hardware. With
 the storage hardware assistance, your host performs these operations faster and 
consumes less CPU, memory, and storage fabric bandwidth.

NAS DISK-LIB:
1. Full /Fast Copy, which is used to copy or migrate data within the same physical array 
2.  RESERVE Space ( Nutanix 3.5)
3. extended statistics.

To verify if nutanix vaai is installed:

~ # esxcli software vib list |grep vaai
nfs-vaai-plugin                1.1-101                             Nutanix   CommunitySupported  2013-05-12

~ # esxcli storage  nfs list
Volume Name   Host              Share   Accessible Mounted  Read-Only  HW Acceleration
-----------------  -----------  ------------------------  -------  ---------  ---------------------
NFS-StorageA   192.168.5.2  /NFS-A   true       true           false             Supported


NFS Uses Full File Clone Primitive and Block storage uses XCOPY primitive. 
Storage vMotion, Hot Cloning cannot leverage  Full File Clone Primitive. 
Full file clone primitive can be leveraged only when a cold migration is initiated.

Reserve Space primitive is implemented  Nutanix 3.5 software version.
More details in
http://cormachogan.com/2012/11/08/vaai-comparison-block-versus-nas/
VMWARE kb-1021976
 Troubleshooting:
VAAI clone will fail:
1. if VM has snapshot
2. if vm is powered on
3. if container name has space or other characters
4. cloning between datastores.

esxtop:
First use ‘u’ to select device view, then enable fields o and p for VAAI stats.

VAAI Logs:

grep -i VAAI var/log/vpxa.log and vmkernel.log

Possible Errors in /var/log/vpxa.log/vmkernel.log
Error:
2012-06-21T01:30:09.165Z [25DAEB90 info 'DiskLib' opID=B2F4E13F-00001118-66]
DISKLIB-LIB   : Failed to create native clone on destination handle :
One of the parameters supplied is invalid (1).


Cause:
  • Cloning to a different datastore
  • Creating a clone from a VM that has a snapshot
Error:


2012-07-18T23:15:49.378Z [4AB51B90 info 'DiskLib' opID=55E1C10D-00000188-3a]
DISKLIB-LIB   : Failed to create native clone on destination handle :
The system cannot find the file specified (25).

Cause:
  • A container name that contains a space
  • A container name that was changed
Error:


2012-07-19T01:59:02.865Z [362ACB90 info 'DiskLib' opID=4A5CFD5E-00006113-5f]
DISKLIB-LIB   : Failed to create native clone on destination handle :
The specified feature is not supported by this version (24).


Cause:
  • Cloning from the local datastore (VMFS)
Also you can vmware.log

2013-07-18T04:09:50.903Z| vcpu-0| SNAPSHOT: SnapshotDumperOpenFromInfo: Creating checkpoint file /vmfs/volumes/78993998-9e053604/ENCmsOLY1007-SCCM/ENCmsOLY1007-Snapshot168.vmsn
2013-07-18T04:09:51.494Z| vcpu-0| nutanix_nfs_plugin: Established VAAI session with NFS server 192.168.5.2
2013-07-18T04:09:51.526Z| vcpu-0| DISKLIB-VMFS  : "/vmfs/volumes/78993998-9e053604/ENCmsOLY1007-SCCM/ENCmsOLY1007-000002-delta.vmdk" : open successful (29) size = 1618106368, hd = 0. Type 8
2013-07-18T04:09:52.255Z| vcpu-0| DISKLIB-LIB   : DiskLibCreateCreateParam: vmfssparse grain size set to : 1