Battery backup unit and flash cache protection

Concept

On this page

A battery or a supercapacitor that lets a hardware RAID controller keep the contents of its write cache through a power cut. A battery keeps the cache memory powered until power returns; a supercapacitor, in newer designs, powers the controller just long enough to copy the cache to flash memory on the card. It makes write-back caching safe, where a write is confirmed as soon as it is in the cache. It also closes the write hole on the controller: the stripe left with new data and old parity when power fails between the two writes.

The risk to the write cache

A RAID controller can have memory of its own, its cache. With write-back caching, the controller confirms a write to the operating system as soon as the data is in that memory, and writes it to the disks later. With write-through, it confirms a write only once the data is on the disks. Broadcom defines the two policies by when "the controller sends a data transfer completion signal to the host": when "the controller cache receives all of the data", or when "the drive subsystem receives all the data".

Write-back makes writes faster, because the operating system does not wait for the disks, and the write penalty of a parity update is paid afterwards. The cost is a window of risk. Until the cached data reaches the disks, it exists only in volatile memory, which loses its contents without power. A power cut in that window loses writes the operating system was told had succeeded: "Since the controller has already acknowledged the I/O as complete, the application is unaware of the data loss."

For a parity array the loss can be worse than the missing writes. If power fails after a stripe's new data reaches the disks but before its new parity does, the stripe is left with data and parity that do not match. This is the write hole: nothing detects the mismatch, and a later rebuild uses the wrong parity and produces wrong data.

Protection methods

Two designs keep the cache through a power cut. They are alternatives: a controller has one or the other.

Battery backup unit (BBU). A battery on the controller keeps the cache memory powered while the machine is off. Broadcom gives the limit as "up to 72 hours, less if the battery is degraded": if power returns within that time, the data is still in the cache and the controller writes it to the disks. A battery wears out, and Broadcom puts its life at "about 1 ½ years". The controller also runs a learn cycle on the battery: a check it repeats periodically, which Broadcom defines as "a battery calibration operation performed periodically by the controller to determine the condition of the battery". In plain terms, the controller measures whether the battery can still do its job. While the battery is missing or discharged, the controller stops using write-back: "the firmware disables the Write Back policy and defaults to the Write Through policy."

Flash cache protection. The newer design does not keep the memory powered. A supercapacitor, a component that stores enough energy to run the card for a short time, powers the controller long enough to copy the cache to the NAND flash memory on the card, a kind of memory that keeps its contents without power. Broadcom sells this design as CacheVault. "Once power returns, the data is restored to the cache and normal operation resumes." In flash the data stays recoverable for "3+ years", against the battery's 72 hours. The supercapacitor needs no scheduled replacement, but it has a learn cycle too: the controller discharges it "from nearly full to a low level" and measures how much charge it can hold. Dell described the same design for its PERC H700 and H800 controllers: on restart the controller will "restore all of the cache data from the flash array to the DRAM array", then "write all cache data to disk".

Completing the interrupted writes

With either design, the writes the controller had confirmed are still there when power returns, and the controller writes them to the disks. An interrupted stripe write is finished, not left half done, so the stripe's data and parity match again. This is how a protected cache closes the write hole. Chen and colleagues stated the general requirement in 1994: "information sufficient to recover the parity must be logged to nonvolatile storage before executing each write operation", and "hardware implementations of RAID systems can implement such logging efficiently using nonvolatile RAM."

A driver running on the CPU has no memory that survives the power cut, so software and firmware RAID (raid engine) close the hole in other ways: Linux md writes to a journal on a separate device or, for RAID 5, keeps a partial parity log in the members' metadata area (write hole). A vendor of software RAID, SoftRAID, recommends an uninterruptible power supply (UPS), a battery for the whole machine, which gives it "time to complete any in-progress writes and shut down gracefully".

Limitations

  • Time. A battery holds the cache for up to 72 hours, fewer as it ages; a power cut that lasts longer loses the data anyway. Flash holds it for 3+ years.
  • Maintenance. A battery has to be replaced, and while it is missing or discharged the controller falls back to write-through, with slower writes.
  • Controller failure. The protected data lives on the controller's own hardware. If the controller fails, the data survives only if its cache can be moved to a replacement. Broadcom's older transportable BBUs were made for this: "A transportable BBU can be used to move to a replacement controller a storage adapter's cached data that has not been written to the disk". The replacement "must be the same model as the failed storage adapter", with a cleared configuration and connected to the same disks.
  • Scope. A protected cache saves the writes that were in flight. It is a power-loss device, not a redundancy device: it does nothing against a failed disk, which is what the array's redundancy is for.

Sources