Changes between Initial Version and Version 1 of LSIraid


Ignore:
Timestamp:
03/05/18 14:50:15 (6 years ago)
Author:
niles
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • LSIraid

    v1 v1  
     1 
     2= Setting up an LSI (MegaRAID) RAID controller card = 
     3 
     4This documents how to set up a RAID using an LSI RAID controller card. 
     5 
     6The first step is to install the software. There are two parts, the GUI and the StorCLI command line tool. At the time of writing the GUI software is available at : 
     7 
     8{{{ https://docs.broadcom.com/docs/17.05.00.02_Linux-64_MSM.gz }}} 
     9 
     10This can be installed (as root) like so : 
     11 
     12{{{ 
     13# tar xfz 17.05.00.02_Linux-64_MSM.gz 
     14# cd disk 
     15./install.csh -s 
     16}}} 
     17 
     18Note that /bin/csh needs to be installed, to do the above steps, so you may have to install tcsh to get that. 
     19 
     20The StorCLI utility is available at : 
     21 
     22{{{ https://docs.broadcom.com/docs/1.21.16_StorCLI.zip }}} 
     23 
     24It is installed on CentOS as follows : 
     25 
     26{{{ 
     27# unzip 1.21.16_StorCLI.zip 
     28# unzip versionChangeSet/univ_viva_cli_rel/storcli_All_OS.zip 
     29# rpm -i storcli_All_OS/Linux/storcli-1.21.06-1.noarch.rpm 
     30}}} 
     31 
     32After the software is installed, you should be able to see the disks attached to 
     33controller zero (/c0) with this command (which can also be used as a general status check 
     34once the RAID is created as detailed here) : 
     35 
     36{{{ 
     37# /opt/MegaRAID/storcli/storcli64 /c0 show 
     38Generating detailed summary of the adapter, it may take a while to complete. 
     39. 
     40. 
     41. 
     42----------------------------------------------------------------------------- 
     43DG Arr Row EID:Slot DID Type  State BT       Size PDC  PI SED DS3  FSpace TR  
     44----------------------------------------------------------------------------- 
     45 0 0   0   41:0     46  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N   
     46 0 0   1   41:1     52  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N   
     47 0 0   2   41:2     53  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N   
     48 0 0   3   41:3     47  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N   
     49 0 0   4   41:4     49  DRIVE Onln  N    7.276 TB dflt N  N   dflt -      N   
     50}}} 
     51 
     52The number 41 is the "enclosure number". This is used when assembling the RAID, or "virtual drive", which is done like so : 
     53 
     54{{{ 
     55[root@netdrms02 ~]# /opt/MegaRAID/storcli/storcli64 /c0 add vd r6 \ 
     56name=SUMS drives=41:0-17 strip=256 Spares=41:18-19  
     57}}} 
     58 
     59NOTE that there is an important "gotcha" : The arguments 
     60above are ORDER DEPENDENT. To hammer this home : the same command 
     61with the "strip" and "drives" entries reversed : 
     62 
     63{{{ 
     64[root@netdrms02 ~]# /opt/MegaRAID/storcli/storcli64 /c0 add vd r6 \ 
     65name=SUMS strip=256 drives=41:0-17 Spares=41:18-19 
     66}}} 
     67 
     68will NOT work, and will blather about not recognizing tokens. 
     69This is a hole that is hard to get out of, since the error was 
     70pretty nonsensical. 
     71 
     72In the above command, "r6" means RAID 6. The online help 
     73is available through 
     74 
     75{{{ 
     76/opt/MegaRAID/storcli/storcli64 /c0 add vd help 
     77}}} 
     78 
     79 
     80In the above command, there are 20 physical disks in the JBOD, we're using  
     81disks 0 to 17 in the raid and setting disks 18 to 19 as spares. 
     82 
     83We also found there was no need to initialize the disk, ie. to tell 
     84the OS about it with a separate command, like : 
     85 
     86{{{ $ /opt/MegaRAID/storcli/storcli64 /c0/v0 start init }}} 
     87 
     88Again, that command fails with an error that is not intuitive. 
     89So time was lost on that. But dmesg showed that the OS already knew about 
     90the disk. 
     91 
     92dmesg showed the new device added as /dev/sdb (the key word below is "Attached") : 
     93 
     94{{{ 
     95[110076.339285] scsi 12:2:0:0: Direct-Access     LSI      MR9286CV-8e      3.27 PQ: 0 ANSI: 5 
     96[110076.359214] sd 12:2:0:0: [sdb] 250031898624 512-byte logical blocks: (128 TB/116 TiB) 
     97[110076.359220] sd 12:2:0:0: [sdb] 4096-byte physical blocks 
     98[110076.359311] sd 12:2:0:0: [sdb] Write Protect is off 
     99[110076.359315] sd 12:2:0:0: [sdb] Mode Sense: 1f 00 00 08 
     100[110076.359320] sd 12:2:0:0: Attached scsi generic sg5 type 0 
     101[110076.359364] sd 12:2:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA 
     102[110076.377336] sd 12:2:0:0: [sdb] Attached SCSI disk 
     103}}} 
     104 
     105OK, spiffy, but we need to use 'parted' to write a gpt partition table for disk of this size. 
     106'fdisk' will not work since it, being a kinds sorta DOS based thing has size limits (another hole 
     107to avoid). So we need to do something like this (for one big ol' partition) : 
     108 
     109{{{ 
     110# parted /dev/sdb 
     111(parted) mklabel gpt 
     112(parted) mkpart primary xfs 0% 100% 
     113(parted) quit 
     114}}} 
     115 
     116OK, now we have /dev/sdb1, so we create an xfs filesystem on it : 
     117 
     118{{{ 
     119# mkfs -t xfs /dev/sdb1 
     120}}} 
     121 
     122Mount it to test : 
     123 
     124{{{ 
     125# mkdir /SUM01; mount -t xfs /dev/sdb1 /SUM01 
     126}}} 
     127 
     128And it shows up in 'df', which is great : 
     129 
     130{{{ 
     131# df -lh 
     132Filesystem               Size  Used Avail Use% Mounted on 
     133/dev/mapper/centos-root   50G  8.3G   42G  17% / 
     134devtmpfs                  63G     0   63G   0% /dev 
     135tmpfs                     63G     0   63G   0% /dev/shm 
     136tmpfs                     63G   18M   63G   1% /run 
     137tmpfs                     63G     0   63G   0% /sys/fs/cgroup 
     138/dev/sda1               1014M  236M  779M  24% /boot 
     139/dev/mapper/centos-home  169G  126M  169G   1% /home 
     140tmpfs                     13G     0   13G   0% /run/user/1000 
     141tmpfs                     13G   12K   13G   1% /run/user/42 
     142tmpfs                     13G  8.0K   13G   1% /run/user/1001 
     143/dev/sdb1                117T   38M  117T   1% /SUM01 
     144}}} 
     145 
     146Un-mount it : 
     147 
     148{{{ 
     149$ umount  /SUM01 
     150}}} 
     151 
     152To get this mount to happen automatically, you have to put something 
     153like this in /etc/fstab : 
     154 
     155{{{ 
     156UUID=2b598aba-0b60-4966-a443-90c9ca730974 /SUM01           xfs     defaults        1 1 
     157}}} 
     158 
     159To get the UUID : 
     160 
     161{{{ 
     162# blkid /dev/sdb1 
     163/dev/sdb1: UUID="2b598aba-0b60-4966-a443-90c9ca730974" TYPE="xfs" PARTLABEL="primary" PARTUUID="5dd86180-2420-4020-ac17-6623a2f6db56"  
     164}}} 
     165 
     166And then it will mount if you ask it to mount everything in /etc/fstab : 
     167 
     168{{{ 
     169# mount -a 
     170}}}