steven at openmedia
May 14, 2009, 2:54 AM
Post #12 of 12
(1786 views)
Permalink
|
On 14/05/2009, at 8:26 PM, Sam Hadley-Jones wrote:
>>
>> I'm really surprised at your freeze up issues. I'm using ext3 and
>> slow
>> deletes and no longer have any playback issues.
>>
>> Did you have a chance to run some performance tests on the raid5
>> array
>> like I suggested as i'm wondering if you have other issues.
>>
>> --------------------------------------------
>> Steven Ellis - Technical Director
>> OpenMedia Limited - The Home of myPVR
>> email - steven [at] openmedia
>> website - http://www.openmedia.co.nz
>>
>
> Here's some test results. A word of warning to any newbies here that
> would
> like to do similar benchmarking; dd is a very powerful command and
> can kill
> data indiscriminately with the most minor typo with no
> confirmation/warning.
>
> Just a recap, this is 5x 500GB Western Digital Green Power SATA
> drives in
> RAID 5 as /dev/md0. Three are connected to the motherboard SATA
> connectors
> and the remaining two to a PCIe SATA card. LVM storage group on top
> with a
> single PV.
>
> mounted:
> /dev/mapper/storage-main on /storage type reiserfs (rw)
> /dev/mapper/storage-tv on /tv type xfs (rw,allocsize=512m)
>
> Raw drive member of array:
>
> sam [at] mediabo:~$ sudo dd if=/dev/sda of=/dev/null bs=1048576 count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 16.8546 s, 63.7 MB/s
>
> Raw RAID5 array:
>
> sam [at] mediabo:~$ sudo dd if=/dev/md0 of=/dev/null bs=1048576 count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 13.9099 s, 77.2 MB/s
>
> LVM LV:
>
> sam [at] mediabo:~$ sudo dd if=/dev/storage/main of=/dev/null bs=1048576
> count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 21.7457 s, 49.4 MB/s
>
> Filesystem on LV 'main' random file read:
>
> sam [at] mediabo:~$ sudo dd if=/storage/videos/Movies/Gangs\ of\ New\
> York.avi
> of=/dev/null bs=1048576 count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 26.1345 s, 41.1 MB/s
>
> Filesystem on LV 'main' write:
>
> sam [at] mediabo:~$ sudo dd of=/storage/testfile if=/dev/zero bs=1048576
> count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 27.2666 s, 39.4 MB/s
>
> Deleting file on 'main':
>
> sam [at] mediabo:~$ time sudo rm /storage/testfile
>
> real 0m1.260s
> user 0m0.008s
> sys 0m1.084s
>
> Filesystem on LV 'tv' write:
>
> sam [at] mediabo:~$ sudo dd of=/tv/testfile if=/dev/zero bs=1048576
> count=10241024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 14.4985 s, 74.1 MB/s
>
> Deleting file on 'tv':
>
> sam [at] mediabo:~$ time sudo rm /tv/testfile
>
> real 0m0.653s
> user 0m0.000s
> sys 0m0.488s
>
> Filesystem on LV 'tv' random file read:
>
> sam [at] mediabo:~$ sudo dd if=/tv/2920_20090514175900.mpg of=/dev/null
> bs=1048576 count=1024
> 1024+0 records in
> 1024+0 records out
> 1073741824 bytes (1.1 GB) copied, 13.2303 s, 81.2 MB/s
>
>
> Conclusion: Inconclusive. The last test showed a higher throughput
> on the
> XFS filesystem than on the underlying LV container. I must admit,
> this is
> not a very controlled test and the backend may have been accessing
> other
> parts of the array at the same time.
>
> That said, I think its safe to assume that the RAID 5 array is a
> serious
> bottleneck. Theoretical speed of the array should be close to 240MB/
> s. I'm
> thinking its either the RAID 5 parity computation or serious latency
> issues
> between the motherboard controller and the PCIe controller.
>
> Others may not see having high throughput as being important. However,
> manipulating this much data, whether it be copying, hashing, etc,
> can take
> hours if not days.
>
Don't expect to see 240MB/s. Your numbers are quite sane. In general I
see raid 5 as 33% faster than the individual drives for a 4 drive array.
Even with hardware raid I seldom see things much faster.
Steven Ellis - Technical Director
OpenMedia Limited
email - steven [at] openmedia
website - http://www.openmedia.co.nz
|
