Growing an mdadm RAID6 array from smaller to larger drives requires a careful sequence of replacing disks, expanding the underlying device, and then extending the logical volume and filesystem. This process is not instantaneous; it involves significant data migration time where the array rebuilds parity information across the new, larger capacity. For a setup with seven 2TB and seven 4TB drives, the operation can take many hours or even days depending on disk speed and system load.
The core challenge lies in ensuring that the LVM physical volumes and the ext4 filesystem recognize the increased size of the mdadm component before attempting to allocate more space. Skipping steps or misordering them can lead to data loss or an inability to expand the storage. The procedure below details how to safely transition your array while maintaining data integrity.
Preparation and Monitoring
Before initiating any hardware changes, you must verify the current state of your array. Use
cat /proc/mdstat to check if all drives are active and synchronized. If the array is already degraded or rebuilding, do not proceed until it has fully recovered. Additionally, ensure you have a recent backup of critical data. While mdadm is robust, power failures or disk errors during the expansion process can corrupt the superblock or LVM metadata.
Check the current size of your RAID device using
mdadm --detail /dev/mdX. Note the "Array Size" field. You will also need to identify which physical disks correspond to which slots in the array using
lsblk or
fdisk -l. It is crucial to know exactly which drive is being replaced to avoid accidentally wiping the wrong disk.
Replacing Drives One by One
You cannot replace all drives at once. You must perform this task sequentially, one drive at a time. Start by identifying a failed or small drive you wish to upgrade. Remove it from the array using:
mdadm /dev/mdX --fail /dev/sdY
mdadm /dev/mdX --remove /dev/sdY
Once removed, physically swap the 2TB drive for a 4TB drive. Ensure the new drive is clean and does not contain partition tables that might confuse mdadm. Insert the new drive and add it to the array:
mdadm /dev/mdX --add /dev/sdZ
The array will now begin rebuilding. Monitor progress via
cat /proc/mdstat. The status will show as "recovery" or "reshape." Do not interrupt this process. Wait until the array returns to a "clean" state with all drives active before proceeding to the next drive. Repeat this process for all 2TB drives until every slot contains a 4TB drive.
Resizing the mdadm Array
After all physical drives have been upgraded to 4TB, the logical array size remains limited to the original 2TB capacity per slot. You must now instruct mdadm to reshape the array to utilize the full space of the new disks. This is done using the
--grow option with the
--size parameter set to
max:
mdadm --grow /dev/mdX --size max
This command triggers a reshape operation. Unlike a simple rebuild, this operation moves data blocks to accommodate the larger sector count. It is I/O intensive and may take several days for large arrays. During this phase, the array is still usable but performance will be significantly degraded. Continue to monitor
/proc/mdstat for the "reshape" status. Once complete, the mdadm device will report the new, larger size.
Extending LVM and Filesystem
Now that the underlying block device has grown, you must extend the Logical Volume (LV) managed by LVM. First, resize the Physical Volume (PV) to match the new mdadm size:
pvresize /dev/mdX
Next, extend your logical volume to consume the newly available space. Assuming your LV is located at
/dev/mapper/vg0-lv_root, use:
lvextend -l +100%FREE /dev/mapper/vg0-lv_root
Finally, resize the ext4 filesystem to fill the expanded logical volume. Run:
resize2fs /dev/mapper/vg0-lv_root
This step is generally fast compared to the previous stages. After completion, verify the new capacity with
df -h. Your storage should now reflect the total capacity of seven 4TB drives minus the overhead for RAID6 parity.
Related Resources
If you encounter issues with filesystem consistency after such large operations, refer to our guide on
Fixing EXT4 Filesystem Superblock Invalid Error. Additionally, understanding how different filesystems interact with mdadm can help in future migrations, as discussed in
Migrating BTRFS Profiles to mdadm RAID.
Frequently Asked Questions
Can I replace all drives simultaneously to save time?
No, you must replace drives one by one. Replacing multiple drives at once risks losing redundancy entirely if a second drive fails during the initial replacement window. Sequential replacement ensures the array remains protected by parity throughout the upgrade process.
How long does the reshape operation take?
The time varies based on disk speed, seek times, and system load. For 7x4TB drives in RAID6, expect anywhere from 24 hours to several days. There is no reliable way to accelerate this beyond using faster disks or reducing background I/O pressure.
What if the array fails to grow after adding new drives?
Ensure that the new drives are correctly added and recognized by
mdadm --detail. Verify that the
--size max flag was used correctly. If the array appears stuck, check kernel logs with
dmesg | tail for error messages related to I/O timeouts or bad sectors.
Is it safe to use the server during the expansion?
Yes, but performance will be severely impacted. The reshape and rebuild processes require heavy read/write operations. Avoid running other intensive tasks like video encoding or large database queries during this period to prevent timeouts or corruption.
Do I need to backup data before resizing the filesystem?
While
resize2fs is generally safe, any change to filesystem geometry carries inherent risk. A power failure during the final stage could corrupt the superblock. Always maintain a verified backup of critical data before performing major infrastructure changes.