1 (edytowany przez lg 2012-04-29 11:55:30)

Temat: RAID5 with mdadm

HOWTO ze względu na szersze grono odbiorców wisi na mojej stronie po ang, przeklejka (jeśli musi być w PL to poprawię):


Here is quick HOWTO for creation of RAID 5 with mdadm.

I will add 3 partitions from disks: sdb, sdc, sdd.

Install mdadm:
sudo apt-get update
sudo apt-get install mdadm


Make partitions on every disk you want to add to array, using fdisk:
fdisk /dev/sdb(c,d,..)

As default align is wrong on those damn WD Greens (check google for WD Green partition align),
align it to sector 64 (not 63), using x (advanced mode) and b.

Don't forget to TURN OFF intelli park.
You have to ways to get it done. The easy way and the hard way.
The easy way is using idle3-tools. (sourceforge it!)
The hard way is bootable DOS and WD tool to mess with parks.
Choose wisely.

Choose partition type: fd, write all settings and quit.

Create an array md0 formed from /dev/sdb1 /dev/sdc1 /dev/sdd1:
mdadm --create /dev/md0 --chunk=1024 --level=5 --raid-devices=3 /dev/sdb1 /dev/sdc1 /dev/sdd1


Assemble your array:
mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1

or use:
mdadm --assemble --scan


While resyncing array is transparent you can use whit it's syncing.

Create filesystem on a new array:
mkfs.ext4 /dev/md0


mdadm configuration is stored in /etc/mdadm.conf or /etc/mdadm/mdadm.conf (depends on distribution)
It can be written there using following command:
sudo mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Add your new RAID to fstab:
/dev/md0      /media/raid     ext4    defaults    1 2


Some peace of advice:

If your machine got RESET or similar "no power" accident, your array will be run as md127 (read only).
After reboot stop the md127, delete it and come back to md0:
mdadm --stop /dev/md127
rm /dev/md127
mdadm --assemble --scan



Tune your RAID!
The defaults settings probably won't satisfy your needs, tune them.

Example script for tuning write and read performance, check times:

#!/bin/bash
# Tuning stripe_cache_size (write performance)
echo Tests for stripe_cache_size:
echo 256 > /sys/block/md0/md/stripe_cache_size ; echo 256 ; sync;echo 3 > /proc/sys/vm/drop_caches; dd if=/dev/zero of=file bs=1M count=30730 ; rm file
echo 512 > /sys/block/md0/md/stripe_cache_size ; echo 512 ; sync;echo 3 > /proc/sys/vm/drop_caches; dd if=/dev/zero of=file bs=1M count=30730 ; rm file
echo 1024 > /sys/block/md0/md/stripe_cache_size ; echo 1024 ; sync;echo 3 > /proc/sys/vm/drop_caches ; dd if=/dev/zero of=file bs=1M count=30730 ; rm file
echo 2048 > /sys/block/md0/md/stripe_cache_size ; echo 2048 ; sync;echo 3 > /proc/sys/vm/drop_caches ; dd if=/dev/zero of=file bs=1M count=30730 ; rm file
echo 4096 > /sys/block/md0/md/stripe_cache_size ; echo 4096 ; sync;echo 3 > /proc/sys/vm/drop_caches ; dd if=/dev/zero of=file bs=1M count=30730 ; rm file
echo 8192 > /sys/block/md0/md/stripe_cache_size ; echo 8192 ; sync;echo 3 > /proc/sys/vm/drop_caches ; dd if=/dev/zero of=file bs=1M count=30730 ; rm file
echo 16384 > /sys/block/md0/md/stripe_cache_size ; echo 16384 ; sync;echo 3 > /proc/sys/vm/drop_caches ; dd if=/dev/zero of=file bs=1M count=30730 ; rm file
echo 32768 > /sys/block/md0/md/stripe_cache_size ; echo 32768 ; sync;echo 3 > /proc/sys/vm/drop_caches ; dd if=/dev/zero of=file bs=1M count=30730

# Tuning Read Ahead (read performance)
echo Tests for blockdev setra:
blockdev --setra 1536 /dev/md0 ; echo 1536 ; sync;echo 3 > /proc/sys/vm/drop_caches; dd if=file of=/dev/null bs=1M
blockdev --setra 4096 /dev/md0 ; echo 4096 ; sync;echo 3 > /proc/sys/vm/drop_caches; dd if=file of=/dev/null bs=1M
blockdev --setra 32768 /dev/md0 ; echo 32768 ; sync;echo 3 > /proc/sys/vm/drop_caches; dd if=file of=/dev/null bs=1M
blockdev --setra 262144 /dev/md0 ; echo 262144 ; sync;echo 3 > /proc/sys/vm/drop_caches; dd if=file of=/dev/null bs=1M
blockdev --setra 524288 /dev/md0 ; echo 524288 ; sync;echo 3 > /proc/sys/vm/drop_caches; dd if=file of=/dev/null bs=1M


Settings which gave me best performance:
blockdev --setra 262144 /dev/sd[bcd]
echo 30730 > /sys/block/md0/md/stripe_cache_size
echo 8192 > /sys/block/md0/md/stripe_cache_active
echo 50000 > /proc/sys/dev/raid/speed_limit_min
echo 1024 > /sys/block/sdb/queue/read_ahead_kb
echo 1024 > /sys/block/sdc/queue/read_ahead_kb
echo 1024 > /sys/block/sdd/queue/read_ahead_kb
echo 256 > /sys/block/sdb/queue/nr_requests
echo 256 > /sys/block/sdc/queue/nr_requests
echo 256 > /sys/block/sdd/queue/nr_requests
echo 0 > /sys/block/sdb/device/queue_depth
echo 0 > /sys/block/sdc/device/queue_depth
echo 0 > /sys/block/sdd/device/queue_depth

golcz.pl