Nov 9, 2014 - I am stuck with a Dell PERC H310 controller with terrible I/O. I have read of the LSI firmware which increases the queue depth and thereby. Dell PowerEdge 12G Latest Firmware. EXE, Recommended, 20.13.3-0001, A11, windows dup, Dell PERC H310 Embedded firmware release 20.13.3-0001.
I’ve been lucky enough to acquire a second-hand Dell H310 RAID controller card, which I’ll be using in my future ZFS build. Let’s start off with some key specs:
- Supported RAID modes: 0, 1, 5, 10, 50
- Interface: PCIe 2.0 x8
- Internal Connectors: 2 x SFF-8087 mini-SAS
- Chipset: LSI SAS 2008 (more info)
For those interested: an extensive spec sheet of the H310 can be found here. The fact that this card contains an LSI I/O Controller will allow us to do some fun stuff with it. There are actually tons of OEM models who offer a far cheaper solution than purchasing an original LSI 9211-8i card. Besides the Dell H310, another good card is the IBM M1015 which can be found all over eBay and offers a great alternative to the H310. I for one chose the H310 over the M1015 as the former has its mini-SAS ports placed vertically, not horizontally like the M1015 does. This might seem like a minor detail, but I really do prefer having them placed vertically, especially when you have to run 2 forward breakout cables through your (possibly very compact) server case. Having the ports placed at the very end of the PCIe card is another perk in my opinion, as it brings you closer to the backplane of your case and it leaves you some room for cable management.
IR firmware vs IT firmware
Most HBA’s are usually shipped with the default IR or Integrated RAID firmware which will allow you to use the most common RAID forms like RAID 0, 1, 5, etc. The firmware Initiator Target (IT) on the other hand will simply use the card in passthrough mode, it does not have an extra RAID layer. Or as LSI gently puts it: “IT firmware maximizes the connectivity and performance aspects of the HBA”. Which is pretty much what we’re looking for here as I’m currently using MDADM and am planning on using ZFS in the future, who both make use of software RAID.
Requirements and preparation
Here are a few things you’ll definitely need:
- Your H310 card (pretty essential)
- A Windows or Linux machine with a PCIe (2.0) x1/x4/x8/x16 slot
- a bootable USB drive
- 30 min of your time
As you can see we’ll need a USB drive. It doesn’t have to be anything fancy as the files that we’ll be putting onto the flash drive are roughly 130 MB combined. Hop on over to this link to download Rufus, which is a great tool to create bootable USB drives. Run the .exe, choose the settings of your liking and hit Start. Below are the settings which I’ve used.
We could now grab the latest firmware from Dell’s website and transfer it to our USB drive. I however opted to download a.rar-file which I’ve found on numerous blog posts and fora and contains all the necessary tools and firmware that we’ll need. Transfer the contents of this .rar file to your USB drive and reboot.
Detecting your card & getting the SAS-address
![Perc h310 firmware upgrade Perc h310 firmware upgrade](http://mywiredhouse.net/wp-content/uploads/2013/09/perc_h310.jpg)
Upon rebooting you should be greeted with a black screen that reads:
PowerEdge Expandable RAID Controller BIOS
Copyright(c) 2011 LSI Corporation
Press <CTRL><R> to Run Configuration Utility
F/W
Initializing Devices 0%
This is actually the BIOS loading screen which goes through the initial setup of the card. The next thing you should do is to boot up into your DOS partition. The stick which we formatted earlier should have 2 partitions: a non-UEFI one (regular DOS) and a UEFI-one. For now we’ll just be using the non-UEFI one. Once you’ve successfully loaded into DOS we can play around with some of the tools that are provided by LSI.
The first thing we’ll input is
MegaRec -adpList
.This displays a list of all the identified controllers, identified by a certain number (MR Card x). If you only have one card installed, it should display MR Card 0. Now that we’ve made sure that our card is being detected, we can request some information about the card, including the SAS address which we’ll need later on. We can do this by entering:
MegaCli -AdpAllInfo -a0
This will display all of the parameters of the specified card. If we want to filter out the SAS address you can either add a pipe so you can execute multiple commands at once as such:
MegaCli -AdpAllInfo -a0 | find 'SAS Address'
Using this pipe symbol somehow didn’t work for me though, so you can always export the output to a seperate text file on your USB drive, and then search for the SAS address manually by booting into your OS:
MegaCli -AdpAllInfo > controller-info.txt
Out with the old: clear SBR and flash memory
In order to clear the SBR (Serial Boot Rom) and flash memory of our card we’ll be using the MegaRec tool once more. Before you delete the SBR, it might be a good idea to backup the original one that came with your card:
MegaRec -readsbr 0 backup.sbr
You can now safely clear the SBR:
MegaRec -writesbr 0 empty.bin
Next up, we’re going to erase the flash memory:
MegaRec -cleanflash 0
Once this has completed successfully you should definitely reboot to make sure everything is working properly. Some even advise you to a cold reboot (i.e. completely power off your system and boot it from scratch) instead of simply rebooting. When rebooting you should notice that the BIOS loading screen of our card which I mentioned earlier is no longer being displayed. This is simply because we cleared the SBR which holds the BIOS.
In with the new: flashing our card
Depending on whether you have a Legacy BIOS or a UEFI-based system the actual flashing of the firmware follows a slightly different procedure. If you’re using the former you can simply boot up from the DOS partition on your USB drive. If you’re using UEFI, the sas2flsh commands are executed from a EFI-shell as opposed to the DOS-environement we were in earlier. Since my ASUS P8P67 is a UEFI-based motherboard I had to boot from the UEFI-partition on my USB drive and enter the EFI-shell. If you do try to issue the sas2flsh.exe command on a UEFI-based system you’ll receive the following error:
ERROR: Failed to initialize PAL. Exiting program.
Once you’ve entered the EFI-shell you first have select the correct drive by typing
fsX:
, with X being the number that corresponds with the correct drive that holds all the files which we’ll be needing. You can verify this by typing ls
after you’ve selected a certain drive. Similar to UNIX, this will give you a list of all the files that are inside that working directory. Important thing to note: the EFI-shell uses the sas2flash.efi program, whilst the DOS-version uses sas2flsh command. Depending on which environment you loaded up earlier, you can change out the commands which I’ll be using during the next few steps.Now we can flash our “empty” card with the Dell firmware:
sas2flash -o -f 6GBSAS.FW
The -o parameter is a switch which allows to enter advanced mode. The -f parameter specifies the filename of the firmware which we’ll be flashing from. We’ll be using the 6GBSAS.FW file, which is the official Dell IT firmware. The offical LSI reference guide which explains all the possible parameters in great detail can be found here by the way. This process might take a few minutes. Once it has completed you can either reboot and go back to the EFI-prompt or you can issue this command which resets the card:
sas2flash_p19.efi -o -reset
![Firmware Firmware](/uploads/1/2/5/8/125829768/123244390.jpg)
Now, we flash the card once again, but this time we use the 2118P7.BIN file. This is actually the LSI P7 IT firmware:
sas2flash -o -f 2118P7.BIN
This will most likely you give you the following error, just enter yes:
NVDATA Product ID and Vendor ID do not match. Would you like to flash anyway [y/n]?
You can now either reboot or use the reset command once again. And finally we can flash the card to the LSI P19 IT firmware:
sas2flash_p19.efi -o -f 2118IT.BIN
Finishing up
Nearly there. Make sure to reset your card one final time. We can now verify whether everything has completed properly by using the list parameter:
sas2flash_p19.efi -o -list
You should see something similar to the picture above. The NVDATA Product ID, Board Name and NVDATA Vendor show you that the card is being the detected as a SAS9211-8i. The Firmware Product ID and Firmware Version confirm that we’ve successfully loaded the LSI P19 IT firmware onto the card.
Last thing to do now is to restore the original SAS address which you should’ve written down somewhere as I explained earlier:
sas2flash_p19.efi -o -sasadd <your 16-digit SAS address>
And that’s it, you’ve succesfully flashed your card to IT firmware. You could add the SBR to the card once again by running sas2flash accompanied by the -b parameter and the filename for your BIOS which has a .rom extension. As this wasn’t really necessary for me and it would only increase boot time I chose not to do this.
Advertisements
OEM version of the LSI SAS 9211-8i, such as the Dell H200, H310 or IBM M1015 are quite popular for use with FreeNAS. However, they need to be flashed with a regular LSI firmware to disable their RAID capabilities in order to passthrough the drives directly to the OS.
Here’s how I upgraded my Dell PERC H200, which came with Dell’s A10 firmware (equivalent to LSI SAS 2008 P07 firmware), to LSI P20 firmware. The newer version also has the advantage that drives larger than 2TB are supported. Also, re-flashing allowed me to not flash a boot ROM to the card, speeding up the boot process of my server as my boot disk is connected to the Intel AHCI controller on the mainboard.
Please note that this operation is not supported by Dell or LSI, may void your warranty and could potentially damage the controller. So proceed at your own risk.
Step 1: Downloading old firmware
- Download the firmware for the Dell 6Gbps SAS HBA (this is a variant of the H200 with 8 external ports instead of 8 internal ports) and extract 6GBPSAS.FW from SASHBA_Firmware_6GBPS-SAS-HBA_07.03.06.00_A10_ZPE.exe.
- Download the P07 firmware for the LSI SAS 9211-8i and extract the file 2118it.bin from 9211_8i_Package_For_P7_Firmware_BIOS_Upgrade_on_MSDOS_and_Windows.zip.
- Download the P05 UEFI flasher for the LSI SAS 9211-8i and extract the file sas2flash.efi from EFI_Installer_P5.zip.
- Place these three files into a directory named P07 on a FAT32-formatted USB flash drive.
Step 2: Downloading current firmware
- Download the current firmware for the LSI SAS 9211-8i and extract the file 2118it.bin from 9211_8i_Package_P20_IR_IT_Firmware_BIOS_for_MSDOS_Windows.
- Download the current UEFI flasher for the LSI SAS 9211-8i and extract the file sas2flash.efi from Installer_P20_for_UEFI.zip.
- Place these two files into a directory named P20 on the USB flash drive.
Step 3: Downloading UEFI shell
- Download an x86_64 UEFI shell. I had to use the v1 shell because my server would only show error messages (about failed assertions and files not found).
- Rename the shell to BOOTX64.efi and place it into a directory named BOOT inside a directory named EFI on the USB flash drive.
Step 4: Flashing
- UEFI boot your server off the flash drive
- Type map -b to find the flash drive
- Switch to it, e.g. by entering fs1:
- cd P07
- sas2flash.efi -listall should show you your controller
- sas2flash.efi -c 0 -list shows you details on the controller. Note down the SAS address in case something goes wrong and you need to reprogram the SAS address.
- Erase the old firmware and boot ROM: sas2flash.efi -o -e 6
- Write the Dell 6Gbps firmware: sas2flash.efi -o -f 6GBPSAS.FW
- Write the LSI P07 firmware: sas2flash.efi -o -f 2118it.bin
- cd ..P20
- Write the LSI P20 firmware: sas2flash.efi -o -f 2118it.bin
Notes
At the end of every command (before I rebooted before step 10), I got the message “Failed Reconnecting the EFI Driver. (EFI Error: Not Found)”. It did not seem to affect anything.
Step 7 showed “Erasing Flash Region” and then after a while “ERROR: Erase Flash Operation Failed!”. I simply proceeded and the error did not appear to affect anything.
Step 8 looked like this:
and the controller details after the flash looked like this:
During step 9, I received the message “NVDATA Versions Compatible. NVDATA Product ID and Vendor ID do not match. Would you like to flash anyway [y/n]?”, where I simply hit y and it proceeded flashing. At the end, it said “Firmware Flash Successful! Resetting Adapter… Adapter Reset Failed!”. Looking at the controller details showed lots of errors:
So at this point, I rebooted the machine. Now the details looked all right:
Step 11 worked without a hitch and afterwards the controller details looked like this:
Booting up the server, I now had a Dell H200 that behaved exactly like a LSI SAS 9211-8i. The only difference was that it still reported its name and PCI ID (1028:1f1c) as a Dell 6Gbps SAS Card. FreeNAS didn’t care about that though.
I was using this controller with FreeNAS 9.2.1.9 and kept on getting kernel messages like
It turns out that these are due to a mismatch between firmware and driver version. FreeBSD 9 ships with driver version 16 and FreeBSD 10 includes version 19. Linux currently has version 18. So make sure that in step 11, you always flash the version that matches your operating system’s driver version, don’t blindly go with version P20 or the latest version. On FreeNAS, you can determine the driver version using dmesg | grep “mps0: Firmware”:
In this case, I had no data loss whatsoever and the person who reported this on the FreeBSD mailing list didn’t either, but both LSI and FreeBSD recommend to keep driver and firmware in sync.
Downgrading firmware
As I had to downgrade the firmware, I needed to do sas2flash.efi -o -e 6 before the flashing. Of course, this means that the crossflashing is undone and you’ll have to start over, flashing Dell 6GPBSAS.FW, flashing LSI P07 IT, and finally flashing the version matching your driver.
6GBPSAS.FW and LSI P07 need to be flashed with the LSI P05 sas2flash.efi because later versions will simply refuse when they encounter a NVDATA mismatch. Once that’s done, you can use the latest (P20 at the moment) sas2flash.efi to flash your final firmware version. This also has the advantage that you don’t get these “Adapter Reset Failed” warnings.
Below is a screenshot of the details of the P16 firmware: