5

Linux software RAID6: rebuild slow

view full story
linux-howto

http://serverfault.com – I am trying to find the bottleneck in the rebuilding of a software raid6. ## Pause rebuilding when measuring raw I/O performance # echo 1 > /proc/sys/dev/raid/speed_limit_min # echo 1 > /proc/sys/dev/raid/speed_limit_max ## Drop caches so that does not interfere with measuring # sync ; echo 3 | tee /proc/sys/vm/drop_caches >/dev/null # time parallel -j0 "dd if=/dev/{} bs=256k count=4000 | cat >/dev/null" ::: sdbd sdbc sdbf sdbm sdbl sdbk sdbe sdbj sdbh sdbg 4000+0 records in 4000+0 records out 1048576000 bytes (1.0 GB) copied, 7.30336 s, 144 MB/s [... similar for each disk ...] # (HowTos)

grtyyu's picture
Submitted by grtyyu on 11/12/2012 – Made popular on 11/12/2012

Stories similar to Linux software RAID6: rebuild slow

Hi,

I am trying to measure the speed of reading a given block size using the dd command.

I have tried

sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'

sudo echo 3 > /proc/sys/vm/drop_caches

echo 3 > /proc/sys/vm/drop_caches

echo 3 | sudo tee /proc/sys/vm/drop_caches

and a bunch of other variations but with every try I get

bash: /proc/sys/vm/drop_caches: Permission denied

How do I clear the ram used for buffers/cache in my centos6 openvz container?

It seems like the

I have to change the smp_affinity of a interrupt permanently.

When a hard disk drive in one of my Linux machines failed, I took the opportunity to migrate from RAID5 to a 6-disk software RAID6 array.

At the time of the migration I did not have all 6 drives - more specifically the fourth and fifth (slots 3 and 4) drives were already in use in the originating array, so I created the RAID6 array with a couple of missing devices.

I read on the wiki article https://wiki.archlinux.org/index.php/Ma … erformance, about adjusting `vm.vfs_cache_pressure` and I did the test as suggested http://rudd-o.com/en/linux-and-free-sof … o-fix-thatsync echo 3 > /proc/sys/vm/drop_caches dd if=/dev/zero of=/tmp/testfile count=1 bs=900M

sysctl -w vm.vfs_cache_pressure=100 find / > /dev/null cp /tmp/testfile /tmp/testfile2 time find / &

Sometimes I see my swap file is a bit bigger then usual. I tried sync && echo 3 > /proc/sys/vm/drop_caches to no avail. But when I tried swapoff -a it takes a bit longer but the swap file is smaller after. How can I clear my swap file without using swapoff?

I use the following script (from this blog http://northernmost.org/blog/find-out-what-is-using-your-swap) to know for each process how much swap it is using

#!/bin/bash # Get current swap usage for all running processes # Erik Ljungstrom 27/05/2011 SUM=0 OVERALL=0 for DIR in `find /proc/ -maxdepth 1 -type d | egrep "^/proc/[0-9]"` ; do PID=`echo $DIR | cut -d / -f 3` PROGNAME=`ps -p $PID -o comm

if i play those command :

sync echo 3 > /proc/sys/vm/drop_caches

Will the server be force to read the correct file on disk until i put 0 again ?

Or this work only one time, and if i want to clear cache again, i've to put 0 and 3 again ?

Thanks

Raid0 grub2 bootable install 1 year 8 weeks ago

Ran another method of testing write and read speeds:results:sh-4.2# dd if=/dev/zero of=test bs=1048576 count=2048 2048+0 records in 2048+0 records out 2147483648 bytes (2.1 GB) copied, 87.708 s, 24.5 MB/s (write) sh-4.2# dd if=test of=/dev/null bs=1048576 2048+0 records in 2048+0 records out 2147483648 bytes (2.1 GB) copied, 1.46163 s, 1.5 GB/s (read) sh-4.2# More confusion!EDIT: Problem solved.