Wednesday, February 1, 2012

Troubleshooting storage issues with ESXi5.0

~ # vmkload_mod -l|grep vmfs3
vmfs3 2 316

~ # vmkiscsi-tool -D -l vmhba37

=========Discovery Properties for Adapter vmhba37=========
iSnsDiscoverySettable : 0
iSnsDiscoveryEnabled : 0
iSnsDiscoveryMethod : 0
iSnsHost.ipAddress : ::
staticDiscoverySettable : 0
staticDiscoveryEnabled : 1
sendTargetsDiscoverySettable : 0
sendTargetsDiscoveryEnabled : 1
slpDiscoverySettable : 0
DISCOVERY ADDRESS : 192.168.5.2
STATIC DISCOVERY TARGET
NAME : iqn.2010-06.com.nutanix:shared_data1-516ad8b0
ADDRESS : 192.168.5.2:62000
BOOT : No
LAST ERR : LOGIN: No Errors
STATIC DISCOVERY TARGET
NAME : iqn.2010-06.com.nutanix:shared_data2-3ae58737
ADDRESS : 192.168.5.2:62001
BOOT : No
LAST ERR : LOGIN: No Errors
STATIC DISCOVERY TARGET
NAME : iqn.2010-06.com.nutanix:shared_data4-8d6bc1cd
ADDRESS : 192.168.5.2:62003
BOOT : No
LAST ERR : LOGIN: No Errors
STATIC DISCOVERY TARGET
NAME : iqn.2010-06.com.nutanix:shared_data3-cbdfce84
ADDRESS : 192.168.5.2:62002
BOOT : No
LAST ERR : LOGIN: No Errors
/dev/disks # esxcli storage core path list | egrep "Target Ident|vmhba37"
Runtime Name: vmhba37:C0:T3:L0
Adapter: vmhba37
Target Identifier: 00023d000001,iqn.2010-06.com.nutanix:shared_data3-cbdfce84,t,1
Target Identifier: sata.0:0
Runtime Name: vmhba37:C0:T2:L0
Adapter: vmhba37
Target Identifier: 00023d000001,iqn.2010-06.com.nutanix:shared_data4-8d6bc1cd,t,1
Target Identifier: sata.0:0
Target Identifier: sata.0:0
Runtime Name: vmhba37:C0:T1:L0
Adapter: vmhba37
Target Identifier: 00023d000001,iqn.2010-06.com.nutanix:shared_data2-3ae58737,t,1
Target Identifier: sata.0:0
Runtime Name: vmhba37:C0:T0:L0
Adapter: vmhba37
Target Identifier: 00023d000001,iqn.2010-06.com.nutanix:shared_data1-516ad8b0,t,1
Target Identifier: sata.0:0
Target Identifier: sata.0:0

/dev/disks # esxcli storage core path list | egrep "Target Identifier|Runtime|naa|Device:"|grep -B1 -A1 NUTAN
Runtime Name: vmhba37:C0:T3:L0
Device: t10.NUTANIX_shared_data3_852409da74945ca63d601382a1db06b2c8f6de10
Target Identifier: 00023d000001,iqn.2010-06.com.nutanix:shared_data3-cbdfce84,t,1
--
Runtime Name: vmhba37:C0:T2:L0
Device: t10.NUTANIX_shared_data4_7ffbcad0adc23e030f69935fe1c0dd5c4a18d187
Target Identifier: 00023d000001,iqn.2010-06.com.nutanix:shared_data4-8d6bc1cd,t,1
--
Runtime Name: vmhba37:C0:T1:L0
Device: t10.NUTANIX_shared_data2_e6c1a2244603b85e94955787f6862d54a277290d
Target Identifier: 00023d000001,iqn.2010-06.com.nutanix:shared_data2-3ae58737,t,1
--
Runtime Name: vmhba37:C0:T0:L0
Device: t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7
Target Identifier: 00023d000001,iqn.2010-06.com.nutanix:shared_data1-516ad8b0,t,1
/dev/disks # esxcli storage core path list -d t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7
iqn.1998-01.com.vmware:ntnx-4-0eaa89c5-00023d000001,iqn.2010-06.com.nutanix:shared_data1-516ad8b0,t,1-t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7
UID: iqn.1998-01.com.vmware:ntnx-4-0eaa89c5-00023d000001,iqn.2010-06.com.nutanix:shared_data1-516ad8b0,t,1-t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7
Runtime Name: vmhba37:C0:T0:L0
Device: t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7
Device Display Name: NUTANIX iSCSI Disk (t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7)
Adapter: vmhba37
Channel: 0
Target: 0
LUN: 0
Plugin: NMP
State: active
Transport: iscsi
Adapter Identifier: iqn.1998-01.com.vmware:ntnx-4-0eaa89c5
Target Identifier: 00023d000001,iqn.2010-06.com.nutanix:shared_data1-516ad8b0,t,1
Adapter Transport Details: iqn.1998-01.com.vmware:ntnx-4-0eaa89c5
Target Transport Details: IQN=iqn.2010-06.com.nutanix:shared_data1-516ad8b0 Alias= Session=00023d000001 PortalTag=1
/dev/disks # esxcli storage core path list -d t10.NUTANIX_shared_data2_e6c1a2244603b85e94955787f6862d54a277290d
iqn.1998-01.com.vmware:ntnx-4-0eaa89c5-00023d000001,iqn.2010-06.com.nutanix:shared_data2-3ae58737,t,1-t10.NUTANIX_shared_data2_e6c1a2244603b85e94955787f6862d54a277290d
UID: iqn.1998-01.com.vmware:ntnx-4-0eaa89c5-00023d000001,iqn.2010-06.com.nutanix:shared_data2-3ae58737,t,1-t10.NUTANIX_shared_data2_e6c1a2244603b85e94955787f6862d54a277290d
Runtime Name: vmhba37:C0:T1:L0
Device: t10.NUTANIX_shared_data2_e6c1a2244603b85e94955787f6862d54a277290d
Device Display Name: NUTANIX iSCSI Disk (t10.NUTANIX_shared_data2_e6c1a2244603b85e94955787f6862d54a277290d)
Adapter: vmhba37
Channel: 0
Target: 1
LUN: 0
Plugin: NMP
State: active
Transport: iscsi
Adapter Identifier: iqn.1998-01.com.vmware:ntnx-4-0eaa89c5
Target Identifier: 00023d000001,iqn.2010-06.com.nutanix:shared_data2-3ae58737,t,1
Adapter Transport Details: iqn.1998-01.com.vmware:ntnx-4-0eaa89c5
Target Transport Details: IQN=iqn.2010-06.com.nutanix:shared_data2-3ae58737 Alias= Session=00023d000001 PortalTag=1

~ # esxcfg-scsidevs -A

vmhba37 t10.NUTANIX_shared_data4_7ffbcad0adc23e030f69935fe1c0dd5c4a18d187
vmhba37 t10.NUTANIX_shared_data3_852409da74945ca63d601382a1db06b2c8f6de10
vmhba37 t10.NUTANIX_shared_data2_e6c1a2244603b85e94955787f6862d54a277290d
vmhba37 t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7

But
~ # esxcfg-scsidevs -m

t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7:1 /vmfs/devices/disks/t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7:1 4f

/dev/disks # esxcli storage vmfs extent list
Volume Name VMFS UUID Extent Number Device Name Partition
-------------------- ----------------------------------- ------------- ------------------------------------------------------------------------ ---------
NTNX-4-local-ds-NTNX 4f1748db-b24ddb90-00b1-0025904b0c90 0 t10.ATA_____INTEL_SSDSA2CW300G3_____________________CVPR133400KE300EGN__ 3
NTNX_datastore_1 4f1de5c4-8b7c3ecc-a9ed-0025904b0c98 0 t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7 1


/dev/disks # ls
vml.0100000000435650523133333430304b4533303045474e2020494e54454c20:8
t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7 vml.01000000007368617265645f64617461315f66316630366138383434656161303330303835613161623562564449534b20
t10.NUTANIX_shared_data1_f1f06a8844eaa030085a1ab5bbd6575daa0ea3b7:1 vml.01000000007368617265645f64617461315f66316630366138383434656161303330303835613161623562564449534b20:1

/dev/disks # esxcli storage filesystem list
Mount Point Volume Name UUID Mounted Type Size Free
------------------------------------------------- -------------------- ----------------------------------- ------- ------ ------------- -------------
/vmfs/volumes/4f1748db-b24ddb90-00b1-0025904b0c90 NTNX-4-local-ds-NTNX 4f1748db-b24ddb90-00b1-0025904b0c90 true VMFS-5 294742130688 54638149632
/vmfs/volumes/4f1de5c4-8b7c3ecc-a9ed-0025904b0c98 NTNX_datastore_1 4f1de5c4-8b7c3ecc-a9ed-0025904b0c98 true VMFS-3 2195533594624 1080925618176
/vmfs/volumes/4ee7c1ed-df35b248-89f3-0025902ef1c9 4ee7c1ed-df35b248-89f3-0025902ef1c9 true vfat 4293591040 4278583296
/vmfs/volumes/0663f37a-83c2d90f-7056-e4ffaac98b5b 0663f37a-83c2d90f-7056-e4ffaac98b5b true vfat 261853184 114044928
/vmfs/volumes/ec72867e-e39b1732-b736-44c76538fffd ec72867e-e39b1732-b736-44c76538fffd true vfat 261853184 114515968
/vmfs/volumes/4ee7c1e5-15813146-c506-0025902ef1c9 4ee7c1e5-15813146-c506-0025902ef1c9 true vfat 299712512 114843648



Resolution Was:
Connect Vcenter, click on Devices in Storage Config, right click on unmounted disk
and then attach it.
Then Rescan VMFS volumes will be in unmounted state, right click and mount it.