Write hole

The gap between writing a stripe's data and writing its parity. A power cut in that gap leaves a parity that no longer matches its data, and nothing in the array notices until a rebuild uses the wrong parity to recompute a missing disk. It affects the parity levels on any engine whose writes are not protected; a battery-backed cache, a journal, or a partial parity log closes it.

The mechanism

Updating a parity stripe takes more than one disk operation: the data block and the parity block are on different disks and are written separately (write penalty). If the power fails after one lands and before the other, the stripe is left in a state no single write would produce. The kernel's own description is short: "for RAID 4/5/6 array, an unclean shutdown can cause data in some stripes to not be in consistent state, eg, data and parity don't match."

On its own this is not yet data loss. The data block is what was written, or what was there before; the parity is stale. As long as every disk is present the array never reads the parity to answer a request, and a resync after the unclean shutdown recomputes it from the data. The hole opens when a disk is missing. Then the parity is not a spare copy, it is the only way to recover the missing block, and a wrong parity yields a wrong block: "data calculated from parity for array blocks that have not been touched by a write request during the unclean shutdown can be incorrect." The block that was never written to comes back corrupted, silently, because the rebuild trusts the parity it finds. And in a degraded array there is "no way to recalculate parity, because one of the disks is missing".

Two things have to coincide: an interrupted write and a missing disk, in either order. That is why the hole is rarely seen and why it is dangerous when it is: a power cut during a rebuild is exactly the coincidence.

Why the array cannot see it

A parity stripe carries no checksum of itself. XOR can tell that a stripe is inconsistent, by recomputing the parity and comparing, but it cannot tell which block is wrong, and it is never consulted on a read unless a disk is gone. Only a deliberate pass over the whole array finds the mismatches: scrubbing, which md exposes as a check that counts them and a repair that rewrites the parity from the data.

What closes it

• A protected write cache. On a hardware controller the whole stripe update sits in cache until every part is on disk, and a BBU or flash module keeps that cache through the cut; when power returns the write is finished, not left half done. This is why the raid engine with such a cache can be silent about the hole, and why the same array on an engine without one cannot.
• A journal. Linux md can put a fast device in front of a RAID 4/5/6 array as a write-through cache: "the write-through cache will cache all data on cache disk first. After the data is safe on the cache disk, the data will be flushed onto RAID disks." An interrupted stripe is replayed from the journal.
• A partial parity log. For RAID 5, md can record, before dispatching a write, the "partial parity" of the stripe, enough to make "parity for the stripe consistent with its state before the write operation, regardless of which chunk writes have completed"; then "if one of the not modified data disks of this stripe is missing, this updated parity can be used to recover its contents". The log lives on the member disks themselves, needs no extra device, and is enabled with --consistency-policy=ppl.
• Power that does not fail. An uninterruptible power supply keeps the machine up long enough to finish its writes. This is the fallback of software and firmware RAID, whose engine is a driver with no memory of its own to protect. Intel's own note for its firmware RAID says what happens otherwise: after a dirty shutdown, parity "may be inconsistent with the data it is supposed to be protecting" and must be recomputed for every stripe.

Mirrors are not immune to an interrupted write, but they fail more gently: the copies may disagree, and a resync picks one, but there is no computed block that could give a wrong answer about a third disk.