Monday to Saturday, 8:00 – 19:00Free collection across Malta and GozoMaltiWhatsApp+356 7933 2877

NAS and RAID

Reassembling a Linux mdadm RAID without making it worse

Linux software RAID (mdadm) sits under most NAS units and many servers. When an array refuses to start, the right commands can bring it back in minutes, and the wrong one can overwrite the very metadata that describes it. This guide is for administrators who want to understand what they are looking at before typing anything.

Published 3 min readBy Mirush Murataj

Cover of the RecData Malta guide: Reassembling a Linux mdadm RAID without making it worse
Read before you write: mdadm --examine on every member, and note the result.

In short

  • Read before you write: mdadm --examine on every member, and note the result.
  • --assemble --force is a calculated risk; --create is a last resort that can destroy data.
  • Work on clones or copy-on-write overlays, never on the only copy.

Rule number one: copy before you experiment

If the data matters, image every member first (with GNU ddrescue on a healthy machine, or in a lab if a disk is failing). If you cannot clone, at least work through copy-on-write overlays: device-mapper snapshots that send every write to a separate file, so the original disks are never modified. Every experiment below should happen on clones or overlays.

Read the superblocks

Each member of an md array carries a superblock describing the array. Start with:

  • cat /proc/mdstat to see what the kernel has assembled, if anything.
  • mdadm --examine /dev/sdX1 on every member partition.

Note for each member:

FieldWhy it matters
Array UUIDAll members of one array share it
Raid Level, Raid DevicesConfirms the layout
Chunk Size, LayoutStripe size and parity rotation
Device RoleThe position of this disk in the array
EventsA counter increased on each array change: the most important clue
Update TimeWhen this member was last written
Array StateHow this member saw the array the last time it was updated

Reading the event counter

Members with the same (or nearly the same) event count are in sync. A member far behind dropped out earlier and holds stale data. In a RAID 5 with four disks where one member is thousands of events behind and another is a few events behind, the best assembly usually uses the three most recent members and leaves the stale one out.

Assembling read-only

Try first: mdadm --assemble --readonly /dev/md0 followed by the members you have chosen. If mdadm refuses because the event counts differ slightly, --force tells it to accept the most recent members and update the lagging superblock. That is a write: do it on overlays, and only when the difference is small and you understand why.

Then mount the file system read-only (for ext4, mount -o ro; for XFS add norecovery so the log is not replayed) and copy the data off before doing anything else.

Why --create is dangerous

mdadm --create with --assume-clean writes new superblocks with the parameters you give. If the chunk size, layout, member order, metadata version or data offset differs from the original even slightly, the array assembles into garbage, and the original superblocks are gone. Newer mdadm versions also use a different default data offset than older ones, a classic trap on arrays created years ago. It is a last resort for specialists, on overlays, with the old --examine output in front of you.

When to stop

  • A member makes noise, disappears from the bus or shows rapidly growing reallocated or pending sectors.
  • Two members have very different event counts and you are not sure which one is current.
  • The file system will not mount read-only.

At that point, every extra attempt reduces what can be recovered. Our RAID data recovery service rebuilds the array from clones, in software, without touching your disks. See also RAID 5 rebuild failed: what happened and what to do now.

Frequently asked questions

Is mdadm --assemble --force safe?

It is relatively safe when the event counts differ by a few units and the disks are healthy, but it writes to the superblocks. Use overlays or clones whenever the data matters.

What does a big difference in the Events field mean?

That member left the array long before the others. Its data is out of date and should normally be left out of the assembly.

Is your data important?

Do not try anything that could make the fault worse. The diagnosis is free and you get a fixed quote before any work starts.

Request a free diagnosis RAID data recovery

Your device has stopped responding? Switch it off and talk to us.

WhatsAppFree diagnosis