Disks fail after ATTO bridge reboot
Applies to
- Fabric MetroCluster
- ONTAP 9
- ATTO FB7500N & FB7600N models using both FC ports
Issue
- After an ATTO bridge is rebooted, multiple disks are reported as missing and fail
Thu Sep 23 17:10:19 +0300 [fsrumogcsh0151: disk_server_1: disk.IO.status:debug]: params: {'deviceName': 'FC-Sw-A1:2.126L6', 'returnCode': '2', 'pathRetryCount': '0', 'adapterStatus': '0xd', 'cdb': 'XXXXX', 'basicTimeout': '9', 'iASCQ': '0x0', 'iSenseKey': '0x0', 'sSenseCode': '', 'ETime': '6866', 'iASC': '0x0', 'victimRetryCount': '14', 'sSenseKey': 'SCSI:no sense', 'targetStatus': '0x0', 'disk_information': 'Disk FC-Sw-A1:2.126L6 Shelf 0 Bay 5 [NETAPP X371_S164A960ATE NA53] S/N [XXXXX] UID [00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000]', 'retryCount': '1', 'pathsTried': '0', 'timeoutRetryCount': '1'}
Thu Sep 23 17:10:19 +0300 [fsrumogcsh0151: config_thread: raid.config.filesystem.disk.missing:info]: File system Disk /aggr_data1/plex0/rg0/FC-Sw-A2:3.126L40 Shelf 0 Bay 13 [NETAPP X371_S164A960ATE NA53] S/N [XXXXX] UID [00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000] is missing.
Thu Sep 23 17:10:19 +0300 [fsrumogcsh0151: config_thread: raid.config.filesystem.disk.missing:info]: File system Disk /aggr_data1/plex0/rg0/FC-Sw-A2:3.126L41 Shelf 0 Bay 14 [NETAPP X371_S164A960ATE NA53] S/N [XXXXX] UID [00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000:00000000] is missing.
- One or more plexes are reported as failed
Example:
Thu Sep 23 17:10:19 +0300 [fsrumogcsh0151: config_thread: raid.rg.degraded:notice]: : Raid group /aggr_data1/plex0/rg0 is degraded
Thu Sep 23 17:10:19 +0300 [fsrumogcsh0151: config_thread: raid.vol.mirror.degraded:alert]: Aggregate aggr_data1 is mirrored and one plex has failed. It is no longer protected by mirroring.
Thu Sep 23 17:10:19 +0300 [fsrumogcsh0151: config_thread: callhome.syncm.plex:alert]: Call home for SYNCMIRROR PLEX FAILED
- The corresponding plex fails and the RAID group is degraded
Example:
Aggregate aggr_data1
(online, raid_dp, mirror degraded, fast zeroed) (block checksums)
Plex /aggr_data1/plex0 (offline, failed, inactive, pool0)
RAID group /aggr_data1/plex0/rg0 (partial, block checksums)
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity FC-Sw-A1:2.126L1 1c 0 0 FC:A 0 SSD N/A 915465/1874872832 915715/1875385008
parity FC-Sw-A2:2.126L2 1b 0 1 FC:A 0 SSD N/A 915465/1874872832 915715/1875385008
data FAILED N/A 915465/ -
data FAILED N/A 915465/ -
data FAILED N/A 915465/ -
data FAILED N/A 915465/ -
data FC-Sw-A1:3.126L47 1c 0 20 FC:B 0 SSD N/A 915465/1874872832 915715/1875385008
data FC-Sw-A1:3.126L48 1a 0 21 FC:B 0 SSD N/A 915465/1874872832 915715/1875385008
data FC-Sw-A2:2.126L23 1b 0 22 FC:A 0 SSD N/A 915465/1874872832 915715/1875385008
data FC-Sw-A1:3.126L50 1a 0 23 FC:B 0 SSD N/A 915465/1874872832 915715/1875385008
Raid group is missing 4 disks.