Changes between Initial Version and Version 1 of LSIraid

Timestamp:

03/05/18 14:50:15 (7 years ago)

Author:

niles

Comment:

--

LSIraid

@@ +1 @@
+
+= Setting up an LSI (MegaRAID) RAID controller card =
+
+This documents how to set up a RAID using an LSI RAID controller card.
+
+The first step is to install the software. There are two parts, the GUI and the StorCLI command line tool. At the time of writing the GUI software is available at :
+
+{{{ https://docs.broadcom.com/docs/17.05.00.02_Linux-64_MSM.gz }}}
+
+This can be installed (as root) like so :
+
+{{{
+# tar xfz 17.05.00.02_Linux-64_MSM.gz
+# cd disk
+./install.csh -s
+}}}
+
+Note that /bin/csh needs to be installed, to do the above steps, so you may have to install tcsh to get that.
+
+The StorCLI utility is available at :
+
+{{{ https://docs.broadcom.com/docs/1.21.16_StorCLI.zip }}}
+
+It is installed on CentOS as follows :
+
+{{{
+# unzip 1.21.16_StorCLI.zip
+# unzip versionChangeSet/univ_viva_cli_rel/storcli_All_OS.zip
+# rpm -i storcli_All_OS/Linux/storcli-1.21.06-1.noarch.rpm
+}}}
+
+After the software is installed, you should be able to see the disks attached to
+controller zero (/c0) with this command (which can also be used as a general status check
+once the RAID is created as detailed here) :
+
+{{{
+# /opt/MegaRAID/storcli/storcli64 /c0 show
+Generating detailed summary of the adapter, it may take a while to complete.
+.
+.
+.
+-----------------------------------------------------------------------------
+DG Arr Row EID:Slot DID Type  State BT       Size PDC  PI SED DS3  FSpace TR 
+-----------------------------------------------------------------------------
+ 0 0   0   41:0     46  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N  
+ 0 0   1   41:1     52  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N  
+ 0 0   2   41:2     53  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N  
+ 0 0   3   41:3     47  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N  
+ 0 0   4   41:4     49  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N  
+}}}
+
+The number 41 is the "enclosure number". This is used when assembling the RAID, or "virtual drive", which is done like so :
+
+{{{
+[root@netdrms02 ~]# /opt/MegaRAID/storcli/storcli64 /c0 add vd r6 \
+name=SUMS drives=41:0-17 strip=256 Spares=41:18-19 
+}}}
+
+NOTE that there is an important "gotcha" : The arguments
+above are ORDER DEPENDENT. To hammer this home : the same command
+with the "strip" and "drives" entries reversed :
+
+{{{
+[root@netdrms02 ~]# /opt/MegaRAID/storcli/storcli64 /c0 add vd r6 \
+name=SUMS strip=256 drives=41:0-17 Spares=41:18-19
+}}}
+
+will NOT work, and will blather about not recognizing tokens.
+This is a hole that is hard to get out of, since the error was
+pretty nonsensical.
+
+In the above command, "r6" means RAID 6. The online help
+is available through
+
+{{{
+/opt/MegaRAID/storcli/storcli64 /c0 add vd help
+}}}
+
+
+In the above command, there are 20 physical disks in the JBOD, we're using 
+disks 0 to 17 in the raid and setting disks 18 to 19 as spares.
+
+We also found there was no need to initialize the disk, ie. to tell
+the OS about it with a separate command, like :
+
+{{{ $ /opt/MegaRAID/storcli/storcli64 /c0/v0 start init }}}
+
+Again, that command fails with an error that is not intuitive.
+So time was lost on that. But dmesg showed that the OS already knew about
+the disk.
+
+dmesg showed the new device added as /dev/sdb (the key word below is "Attached") :
+
+{{{
+[110076.339285] scsi 12:2:0:0: Direct-Access     LSI      MR9286CV-8e      3.27 PQ: 0 ANSI: 5
+[110076.359214] sd 12:2:0:0: [sdb] 250031898624 512-byte logical blocks: (128 TB/116 TiB)
+[110076.359220] sd 12:2:0:0: [sdb] 4096-byte physical blocks
+[110076.359311] sd 12:2:0:0: [sdb] Write Protect is off
+[110076.359315] sd 12:2:0:0: [sdb] Mode Sense: 1f 00 00 08
+[110076.359320] sd 12:2:0:0: Attached scsi generic sg5 type 0
+[110076.359364] sd 12:2:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
+[110076.377336] sd 12:2:0:0: [sdb] Attached SCSI disk
+}}}
+
+OK, spiffy, but we need to use 'parted' to write a gpt partition table for disk of this size.
+'fdisk' will not work since it, being a kinds sorta DOS based thing has size limits (another hole
+to avoid). So we need to do something like this (for one big ol' partition) :
+
+{{{
+# parted /dev/sdb
+(parted) mklabel gpt
+(parted) mkpart primary xfs 0% 100%
+(parted) quit
+}}}
+
+OK, now we have /dev/sdb1, so we create an xfs filesystem on it :
+
+{{{
+# mkfs -t xfs /dev/sdb1
+}}}
+
+Mount it to test :
+
+{{{
+# mkdir /SUM01; mount -t xfs /dev/sdb1 /SUM01
+}}}
+
+And it shows up in 'df', which is great :
+
+{{{
+# df -lh
+Filesystem               Size  Used Avail Use% Mounted on
+/dev/mapper/centos-root   50G  8.3G   42G  17% /
+devtmpfs                  63G     0   63G   0% /dev
+tmpfs                     63G     0   63G   0% /dev/shm
+tmpfs                     63G   18M   63G   1% /run
+tmpfs                     63G     0   63G   0% /sys/fs/cgroup
+/dev/sda1               1014M  236M  779M  24% /boot
+/dev/mapper/centos-home  169G  126M  169G   1% /home
+tmpfs                     13G     0   13G   0% /run/user/1000
+tmpfs                     13G   12K   13G   1% /run/user/42
+tmpfs                     13G  8.0K   13G   1% /run/user/1001
+/dev/sdb1                117T   38M  117T   1% /SUM01
+}}}
+
+Un-mount it :
+
+{{{
+$ umount  /SUM01
+}}}
+
+To get this mount to happen automatically, you have to put something
+like this in /etc/fstab :
+
+{{{
+UUID=2b598aba-0b60-4966-a443-90c9ca730974 /SUM01           xfs     defaults        1 1
+}}}
+
+To get the UUID :
+
+{{{
+# blkid /dev/sdb1
+/dev/sdb1: UUID="2b598aba-0b60-4966-a443-90c9ca730974" TYPE="xfs" PARTLABEL="primary" PARTUUID="5dd86180-2420-4020-ac17-6623a2f6db56" 
+}}}
+
+And then it will mount if you ask it to mount everything in /etc/fstab :
+
+{{{
+# mount -a
+}}}