Skip to content

Instantly share code, notes, and snippets.

@weldpua2008

weldpua2008/MegaCLI cheet sheets.md

Last active July 18, 2018 11:52
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save weldpua2008/e0212389a2b4c7dbf04dd449439dca3d to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save weldpua2008/e0212389a2b4c7dbf04dd449439dca3d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
MegaCLI cheet sheets.md

HOW TO ADD / REPLACE DISK

  1. Find out list of physical disks installed in server
# Download https://github.com/npinto/megaclisas-status/blob/master/megaclisas-status
/opt/MegaRAID/MegaCli/megaclisas-status
-- Controller information --
-- ID | H/W Model                  | RAM    | Temp | BBU    | Firmware
c0    | AVAGO MegaRAID SAS 9361-8i | 1024MB | 58C  | Good   | FW: 24.15.0-0021
 
-- Array information --
-- ID | Type   |    Size |  Strpsz |   Flags | DskCache |   Status |  OS Path | CacheCade |InProgress
c0u0  | RAID-1 |   3637G |  256 KB |   RA,WB |  Default |  Optimal | /dev/sda | None      |None
c0u1  | RAID-0 |   3637G |  256 KB |   RA,WB |  Default |  Optimal | /dev/sdb | None      |None
c0u2  | RAID-0 |   3637G |  256 KB |   RA,WB |  Default |  Optimal | /dev/sdc | None      |None
c0u3  | RAID-0 |   3637G |  256 KB |   RA,WB |  Default |  Optimal | /dev/sdd | None      |None
c0u4  | RAID-0 |   3637G |  256 KB |   RA,WB |  Default |  Optimal | /dev/sde | None      |None
c0u5  | RAID-0 |   3637G |  256 KB |   RA,WB |  Default |  Optimal | /dev/sdl | None      |None
c0u6  | RAID-0 |   3637G |  256 KB |   RA,WB |  Default |  Optimal | /dev/sdg | None      |None
c0u7  | RAID-0 |   3637G |  256 KB |   RA,WB |  Default |  Optimal | /dev/sdh | None      |None
c0u8  | RAID-0 |   3637G |  256 KB |   RA,WB |  Default |  Optimal | /dev/sdi | None      |None
c0u9  | RAID-0 |   3637G |  256 KB |   RA,WB |  Default |  Optimal | /dev/sdj | None      |None
c0u10 | RAID-0 |    894G |  256 KB | ADRA,WB |  Default |  Optimal | /dev/sdk | None      |None
 
-- Disk information --
-- ID   | Type | Drive Model                                    | Size     | Status          | Speed    | Temp | Slot ID  | LSI Device ID
c0u0p0  | HDD  | WD-WCC132249874WDC WD4000FYYZ-01UL1B1 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 21C  | [8:0]    | 9
c0u0p1  | HDD  | WD-WCC132300526WDC WD4000FYYZ-01UL1B1 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 20C  | [8:1]    | 15
c0u1p0  | HDD  | WD-WCC135HU56E8WDC WD4000FYYZ-01UL1B2 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 20C  | [8:2]    | 10
c0u2p0  | HDD  | WD-WMC130E81TH2WDC WD4000FYYZ-01UL1B2 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 20C  | [8:3]    | 16
c0u3p0  | HDD  | WD-WMC130470512WDC WD4000FYYZ-01UL1B1 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 21C  | [8:4]    | 11
c0u4p0  | HDD  | WD-WMC130E1S8TKWDC WD4000FYYZ-01UL1B2 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 21C  | [8:5]    | 20
c0u5p0  | HDD  | WD-WCC135PH0V6HWDC WD4000FYYZ-01UL1B2 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 21C  | [8:6]    | 12
c0u6p0  | HDD  | WD-WMC130356088WDC WD4000FYYZ-01UL1B1 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 21C  | [8:7]    | 19
c0u7p0  | HDD  | WD-WMC130318008WDC WD4000FYYZ-01UL1B1 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 21C  | [8:8]    | 13
c0u8p0  | HDD  | WD-WMC130DA57VZWDC WD4000FYYZ-01UL1B2 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 25C  | [8:9]    | 17
c0u9p0  | HDD  | WD-WCC13HSXP7UNWDC WD4000FYYZ-01UL1B2 01.01K04 | 3.637 TB | Online, Spun Up | 6.0Gb/s  | 23C  | [8:10]   | 14
c0u10p0 | SSD  | 202968AESDLFOCAR-960G-1HST ZZ2CRC92            | 893. Gb  | Online, Spun Up | 6.0Gb/s  | N/A  | [8:11]   | 18
  1. Unmount disks from specific end point
  2. Pick unused disk the one had been replaced or added and create Virtual Disk (If this step fails, proceed to step 4. Then repeat step 3 ):
/opt/MegaRAID/storcli/storcli64 /c0 add vd type=r0 drives=8:11
Controller = 0
Status = Success
Description = Add VD Succeeded

/opt/MegaRAID/storcli/storcli64 /c0 /v10 set name="JBOD-SSD"
  1. Clear foreign controller data(If this step fails, proceed to step 5. Then repeat step 4)
/opt/MegaRAID/MegaCli/MegaCli64 -CfgForeign -Scan -aall
/opt/MegaRAID/MegaCli/MegaCli64 -CfgForeign -Clear -aall
  1. Clear Preserved Cache
# Find Non exist non exist VD = {x}

/opt/MegaRAID/storcli/storcli64 /c0 show
# Do preservedCache for VD
/opt/MegaRAID/storcli/storcli64 /c0/v{x} delete preservedCache
  1. Enable jbod for all
/opt/MegaRAID/storcli/storcli64 /c0 set jbod=on
  1. Change state
/opt/MegaRAID/storcli/storcli64 /c0 /e13 /s8 set jbod
/opt/MegaRAID/storcli/storcli64 /c0 /e13 /s8 set good
# OR
/opt/MegaRAID/storcli/storcli64 /c0 /e13 /s8  set good -force
  1. Re-visit show status command and see if the VS was properly created:
/opt/MegaRAID/storcli/storcli64 /c0 show all
VD LIST :
=======

-----------------------------------------------------------------
DG/VD TYPE State Access Consist Cache Cac sCC Size Name
-----------------------------------------------------------------
0/0 RAID1 Optl RW Yes RWBD - ON 3.638 TB RAID1-A
1/1 RAID0 Optl RW Yes RWBD - ON 3.638 TB JBOD-B
2/2 RAID0 Optl RW Yes RWBD - ON 3.638 TB JBOD-C
3/3 RAID0 Optl RW Yes RWBD - ON 3.638 TB JBOD-D
4/4 RAID0 Optl RW Yes RWBD - ON 3.638 TB JBOD-E
5/5 RAID0 Optl RW Yes RWBD - ON 3.638 TB JBOD-F
6/6 RAID0 Optl RW Yes RWBD - ON 3.638 TB JBOD-G
7/7 RAID0 Optl RW Yes RWBD - ON 3.638 TB JBOD-H
8/9 RAID0 Optl RW Yes RWBD - ON 3.638 TB JBOD-J
9/8 RAID0 Optl RW Yes RWBD - ON 3.638 TB JBOD-I
10/10 RAID0 Optl RW Yes RWBD - ON 893.75 GB```
9. Create partition and filesystem for new VS that was just created:
```bash
#Create partition:
parted /dev/sdk mklabel gpt
parted -a optimal /dev/sdk mkpart primary 0% 100%

#Create FS:
mkfs.ext4 /dev/sdk1

#Get Block Device UUID:
blkid | grep sdk1
/dev/sdk1: UUID="e21b7e1f-9dc9-4d74-94fc-b0bb272bc843" TYPE="ext4"
  1. Take disk UUID and add it into appropriate location under /etc/fstat
  2. Check the disk statuses
 /opt/MegaRAID/storcli/storcli64 /c0 /eall /sall show all | grep -iE "error"



Media Error Count = 0

Other Error Count = 0

Media Error Count = 0

Other Error Count = 0

Media Error Count = 0

Other Error Count = 0

Media Error Count = 0

Other Error Count = 0

Media Error Count = 86

Other Error Count = 0

Media Error Count = 0

Other Error Count = 0```

### HOW TO CHECK RAID & REBUILD
1. Check raid status
```bash 
/opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL
  1. After the drive is replaced Raid will start to rebuild it.
  2. in order to confirm run the command megacli -LDInfo -L2 -a0 | grep -i raid
/opt/MegaRAID/MegaCli/megaclisas-status
# /opt/MegaRAID/MegaCli/MegaCli64   -PDRbld -ShowProg -PhysDrv [32:0] -aALL
Device(Encl-32 Slot-0) is not in rebuild process
 
Exit Code: 0x00
# /opt/MegaRAID/MegaCli/megaclisas-status
-- Controller information --
-- ID | H/W Model      | RAM    | Temp | BBU    | Firmware
c0    | PERC H730 Mini | 1024MB | 60C  | Good   | FW: 25.5.4.0006
 
-- Array information --
-- ID | Type   |    Size |  Strpsz |   Flags | DskCache |   Status |  OS Path | CacheCade |InProgress
c0u0  | RAID-1 |    894G |   64 KB | ADRA,WB |  Default |  Optimal | /dev/sda | None      |None
 
-- Disk information --
-- ID  | Type | Drive Model                                        | Size     | Status          | Speed    | Temp | Slot ID  | LSI Device ID
c0u0p0 | SSD  | S2TZNX0J304400 SAMSUNG MZ7LM960HMJP-00005 GXT5204Q | 893. Gb  | Online, Spun Up | 6.0Gb/s  | 32C  | [32:0]   | 0
c0u0p1 | SSD  | S2HTNX0J501108 SAMSUNG MZ7KM960HAHP-00005 GXM1103Q | 893. Gb  | Online, Spun Up | 6.0Gb/s  | 32C  | [32:1]   | 1

HOW TO DISABLE CacheCade

When IT pushes hard disk drives (HDD) arrays to reach their I/O potential, data “hot spots” become inevitable. Utilizing a small solid state drive (SSD) investment as a front-side flash cache for the much larger disk array, MegaRAID® CacheCade® Pro 2.0 software dynamically keeps the “hottest” data in flash memory. This can dramatically improve read/write speeds and drop latency. Unlock substantial performance gains from HDD arrays with minimal SSD investment Assign and manage the SSD pool with intuitive management software Eliminate or reduce wasted capacity caused by short-stroking hard disk drives Extend HDD longevity by reducing wear from the most commonly accessed data

  1. Check status of CacheCade
]#  /opt/MegaRAID/MegaCli/megaclisas-status
-- Controller information --
-- ID | H/W Model                  | RAM    | Temp | BBU    | Firmware
c0    | AVAGO MegaRAID SAS 9361-8i | 1024MB | 66C  | Good   | FW: 24.15.0-0021
 
-- Array information --
-- ID | Type   |    Size |  Strpsz | Flags | DskCache |   Status |  OS Path | CacheCade |InProgress
c0u0  | RAID-1 |   3637G |  256 KB | RA,WB |  Default |  Optimal | /dev/sda | None      |None
c0u1  | RAID-0 |   3637G |  256 KB | RA,WB |  Default |  Optimal | /dev/sdb | None      |None
c0u2  | RAID-0 |   3637G |  256 KB | RA,WB |  Default |  Optimal | /dev/sdc | None      |None
c0u3  | RAID-0 |   3637G |  256 KB | RA,WB |  Default |  Optimal | /dev/sdd | None      |None
c0u4  | RAID-0 |   3637G |  256 KB | RA,WB |  Default |  Optimal | /dev/sde | None      |None
c0u5  | RAID-0 |   3637G |  256 KB | RA,WB |  Default |  Optimal | /dev/sdf | None      |None
c0u6  | RAID-0 |   3637G |  256 KB | RA,WB |  Default |  Optimal | /dev/sdg | None      |None
c0u7  | RAID-0 |   3637G |  256 KB | RA,WB |  Default |  Optimal | /dev/sdh | None      |None
c0u8  | RAID-0 |   3637G |  256 KB | RA,WB |  Default |  Optimal | /dev/sdi | None      |None
c0u9  | RAID-0 |   3637G |  256 KB | RA,WB |  Default |  Optimal | /dev/sdj | None      |None
c0u10 | RAID-0 |    894G |  256 KB | RA,WB |  Default |  Optimal | /dev/sdk | None      |None
  1. disable for v1
/opt/MegaRAID/storcli/storcli64 /c0/v1 set ssdCaching=off

Related articles https://www.broadcom.com/products/storage/raid-controllers/megaraid-cachecade-pro-software https://www.thomas-krenn.com/en/wiki/MegaRAID_CacheCade_SSD_Cache

EXPAND LOCAL STORAGE

  1. Expand
#storcli64 /c0/v0 start migrate type=raid5 option=add drives=10:6,10:7,10:8,10:9,10:10,10:11 
Controller = 0 Status = Success Description = Start 
MIGRATE Operation Success
  1. Wait until expansition is over
# storcli64 /c0/v0 show migrate Controller = 0 
Status = Success Description = None
VD Operation Status : ===================
-------------------------------------------------------
VD Operation Progress% Status Estimated Time Left
------------------------------------------------------- 
0 Migrate 0 In progress 11 Hours 55 Minutes 
-------------------------------------------------------
  1. Rescan local disk to make sure rebuild is properly done
echo "1"  > /sys/class/scsi_disk/8\:2\:0\:0/device/rescan
  1. Following response will appears under dmesg
#dmesg
sd 8:2:0:0: [sda] 25774915584 512-byte logical blocks: (13.1 TB/12.0 TiB)
sd 8:2:0:0: [sda] 4096-byte physical blocks
sda: detected capacity change from 5998525808640 to 13196756779008
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment