RAID engine

Concept

On this page

The component that turns a RAID layout into disk operations. It owns the array's metadata, translates every request from the array's addresses to the members', writes the copies or computes the parity, and keeps serving when a disk fails. It is a role, not a fixed device: a RAID-on-Chip on a controller card, the operating system's own driver, or a boot firmware working with that driver.

Responsibilities

Every RAID engine has the same jobs:

  • Keep the metadata. Which disks belong to the array, in what order, with what layout, and in what state. Linux md writes it on each member as a superblock; a controller keeps its own records on the disks (MegaRAID writes them in the SNIA DDF format).
  • Translate addresses. Every read and write arrives for the array's sectors and must be turned into the members' sectors, by the rule of the segmentation.
  • Produce the redundancy. Write every copy of a mirrored block, compute the parity of a stripe and write it, keep the two consistent.
  • Survive a failure. When a member dies, keep answering from the survivors (a degraded array) and, given a replacement, rebuild the missing member from what the redundancy holds.
  • Present the result. Upstream, the array is one block device: a virtual drive when a controller presents it, an md device when the operating system does.

The three implementations

The physical disks reach the operating system along one path: backplane, HBA, the PCIe bus, the CPU, the OS. The engine sits on that path. Depending on what runs it, it is called hardware, software or firmware (or fake) RAID.1

Hardware RAID. A controller card carries a RAID-on-Chip (RoC), a processor dedicated to the list above. Broadcom's SAS3916, for one, pairs a dual-core ARM processor with "hardware acceleration engines for RAID 5 and RAID 6 parity calculations" and drives the card's own memory, 4 or 8 GB of write cache on a MegaRAID 9560. The operating system never sees the member disks: what it is given is the controller's virtual drive. In Broadcom's MegaRAID vocabulary, "virtual drives are drive groups or spanned drive groups that are available to the operating system".

Software RAID. No engine object on the path. The HBA passes the disks through and the operating system's driver is the engine: md in Linux (configured with mdadm), Storage Spaces in Windows. The CPU does the computing, the OS sees every member disk and, beside them, the array device it made.

Firmware RAID, also called fake RAID: the card has no RAID engine of its own, and the operating system does the work on the CPU. A disk controller on the motherboard, or on a cheap add-in card, whose boot firmware knows how to read an array: an option ROM in the system BIOS which, in Intel's words, manages the drives and RAID volumes "before the operating system loads". That is what it is for: the machine can boot from the array. The array's metadata is written on the member disks, in the firmware's format. Once the OS is up, its driver does the engine's work on the CPU, exactly as in software RAID: Intel's own description of the Linux boot is that the initial RAM file system loads the MD RAID driver, "and then RAID volumes are assembled". mdadm runs these volumes as md arrays with an external metadata format, imsm, "supported by an option-rom on some platforms".

Distinguishing criterion

A hardware controller and a firmware-RAID card can sit in the same PCIe slot, and on AMD's AM4 platform some of the SATA ports AMD RAID works with come from the processor itself ("processor-direct SATA"), so the controller and the CPU share a die. What separates the three is two questions: who owns the metadata, and who computes.

  • hardware: the controller owns it and computes it
  • firmware: the firmware defines the metadata and reads it at boot; the operating system updates it and computes
  • software: the operating system owns it and computes it

Two consequences follow from where the engine runs.

  • The write hole. A hardware engine's write cache can be protected against power loss, by a BBU or by flash, so a stripe write cut by a power failure is finished when power returns: the controller closes the parity write hole itself. A driver on the CPU instead has no memory that survives the cut, so Linux md closes the hole on the disks: with a write journal on a separate device or, for RAID 5, with a partial parity log kept in the members' metadata area, which Intel-format firmware arrays can use too.
  • The driver. Both kinds need a driver in the operating system, but not the same kind. A hardware controller's driver (megaraid_sas in Linux) carries no RAID logic, because the controller does that work. A firmware array cannot even be assembled without the operating system's RAID driver.

Notes

  1. The three cases are how the sandbox models the engine — a computing chip, a metadata-only chip, or the operating system — and which case a build is in is decided by which object sits on the path, not by where it sits. Real products blur the lines; the sandbox draws the case, not the product. See the model's choices. ↩

Sources