Speeding Up MDADM RAID Rebuilds

Speeding Up MDADM RAID Rebuilds I’m slowly migrating a bunch of awesome things from a really old server, it’s still running Ubuntu 10.04.. to a really nice and shiny one. Which has 2 new 3TB HDDs in RAID 1, which are syncing..


cat /proc/mdstat
md3 : active raid1 sda4[0] sdb4[1]
1847478528 blocks super 1.2 [2/2] [UU]
[>....................] resync = 0.1% (2061952/1847478528) finish=28857.9min speed=1065K/sec

That’s like 40 days of syncing.. surely we can do better than that? Can’t we speed up this MDADM RAID 1 rebuild?

Speeding Up MDADM RAID Rebuilds

Sure we can, we’re awesome, I’m awesome, you’re awesome, can I get a hell yeah? HELL YEAH!

Likes lots of things that are tuneable, we have to do NASTY things to /proc. This bit sets the minimum speed we want mdadm to go at.


cat /proc/sys/dev/raid/speed_limit_min
1000

We’re greedy though and in the words of Tim Tool Time Taylor, MORE POWER. As root..


echo 50000 > /proc/sys/dev/raid/speed_limit_min

We’ve just made mdadm run FIFTY TIMES faster. FOR FREE. We didn’t even swap out the SATA disks for SAS SSDs, we just changed a number ..


md3 : active raid1 sda4[0] sdb4[1]
1847478528 blocks super 1.2 [2/2] [UU]
[=====>...............] resync = 26.0% (480866560/1847478528) finish=141.9min speed=160501K/sec

Now 10 minutes later we’re already over 1/4 the way through, 2 hours left baby. Hell yeah.

Saving the RAID

I managed to screw up my home machine when upgrading from Dapper to Hardy somehow, so decided to flatten and reinstall Hardy from scratch.

This is all well and good but I have a RAID 5 array on my machine which stores all my important documents and stuff (I really hate drive failure). Ubuntu didn’t suddenly detect my RAID array and figure everything out, which was not entirely unsurprising but was a little scary.

After some light reading, and a few sighs and heart flutters I worked out the magic runes to tell the system I have a RAID array, where it was and how to use it!


root@server:/root# mdadm -A /dev/md0 -v /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde
mdadm: looking for devices for /dev/md0
mdadm: /dev/sda is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sdb is identified as a member of /dev/md0, slot 1.
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 2.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 3.
mdadm: /dev/sde is identified as a member of /dev/md0, slot 4.
mdadm: added /dev/sdb to /dev/md0 as 1
mdadm: added /dev/sdc to /dev/md0 as 2
mdadm: added /dev/sdd to /dev/md0 as 3
mdadm: added /dev/sde to /dev/md0 as 4
mdadm: added /dev/sda to /dev/md0 as 0
mdadm: /dev/md0 has been started with 5 drives.
root@server:/root# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sda[0] sde[4] sdd[3] sdc[2] sdb[1]
976793600 blocks level 5, 128k chunk, algorithm 2 [5/5] [UUUUU]
unused devices:
root@server:/root# mdadm -Es
ARRAY /dev/md0 level=raid5 num-devices=5 UUID=19c77b42:c5e86159:c23e7b08:7f8ceabd