exchanging a defective drive in a software RAID

setup: server with configured software RAID1 and LVM. If you have configured mdadm monitoring correctly you might receive an email like the following one day:

This requires immediate action because one of your hard disks is possibly broken.

In this case /dev/sda is the defective device.  For your support ticket you need to provide information that proves that the device is broken.

Next you need to remove the drive from the RAID:

If removing fails with the message “mdadm: hot remove failed for /dev/sdXY: Device or resource busy”, try

first.

Again check mdstat:

Now you are ready and have all information together to open your support ticket. Ensure that you have provided a mail adress in the robot which you have access to even if your server is not reachable.
After the drive has been replaced you might have to install grub again in order to boot your server. You can achieve this from the rescue system.

First copy the partition table and assign a new random UUID:

Mount you root filesystem after detecting your existing volume group / drives. Then write grub to the new drive:

If everything went fine you are now able to log into your system and add the new drive again to the RAID.

Again, check mdstat and hopefully you will see the sync in progress:

Based on http://wiki.hetzner.de/index.php/Festplattenaustausch_im_Software-RAID

This entry was posted in Debian, Hetzner. Bookmark the permalink.