Mailing List Archive: DRBD: Users

Re: drbd on top of hardware raid

Index | Next | Previous | View Threaded

Lars.Ellenberg at linbit

Aug 9, 2006, 1:27 PM

Post #1 of 3 (581 views)
Permalink

Re: drbd on top of hardware raid

/ 2006-08-09 13:53:39 -0600
\ Monty Taylor:
> No, it's an internal LSI MegaRAID configured in Raid5. (not my choice)
>
> On the positive side, though, it supports 90M/sec of reading.

if it is an option, try to put it into jbod,
then do software raid5 on those.
should help if that controller is "CPU" bound,
and your file server is not...

and, btw, you probably know that already ...
raid controllers tend to pretend against the kernel that data was
on stable storage, even when they indeed only have it still in their
write back cache.
if that cache is battery backed, ok.
if not, and you get data corruption after blackout/brownout,
don't blame linux, nor the device/file system drivers,
nor drbd either...

unfortunately sometimes you can not put it in "write through" mode
(use that cache, but only say its done when its really done), but
only switch off the cache completely.
and then your write throughput gets really bad...

so you better make sure you run on some ups, and monitor it,
and power down the box cleanly in case the ups capacity runs out.

--
: Lars Ellenberg Tel +43-1-8178292-0 :
: LINBIT Information Technologies GmbH Fax +43-1-8178292-82 :
: Schoenbrunner Str. 244, A-1120 Vienna/Europe http://www.linbit.com :
__
please use the "List-Reply" function of your email client.
_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user

matthew at sixapart

Aug 9, 2006, 2:37 PM

Post #2 of 3 (551 views)
Permalink

Re: drbd on top of hardware raid [In reply to]

On Aug 9, 2006, at 1:27 PM, Lars Ellenberg wrote:

> / 2006-08-09 13:53:39 -0600
> and, btw, you probably know that already ...
> raid controllers tend to pretend against the kernel that data was
> on stable storage, even when they indeed only have it still in their
> write back cache.
> if that cache is battery backed, ok.
> if not, and you get data corruption after blackout/brownout,
> don't blame linux, nor the device/file system drivers,
> nor drbd either...
>
> unfortunately sometimes you can not put it in "write through" mode
> (use that cache, but only say its done when its really done), but
> only switch off the cache completely.
> and then your write throughput gets really bad...
>
> so you better make sure you run on some ups, and monitor it,
> and power down the box cleanly in case the ups capacity runs out.

Drive manufacturers also like to enable the write caches on their
drives. It's the WCE bit in the disk mode pages. Unfortunately it's
not possible to modify this bit through a megaraid. There's a flag
to temporarily disable it at runtime but rebooting or resetting a
disk will once again set it to write cache. My preferred method is
to yank the drives and use sdparm (http://sg.torque.net/sg/
sdparm.html) through a plain ol scsi controller to write the mode pages.

If you want to test that your disks are behaving nicely or badly you
can use the script referenced here:

http://brad.livejournal.com/2116715.html

I've run that against drbd with a good underlying store and found
drbd to work perfectly with respect to power loss situations.

_______________________________________________
drbd-user mailing list
drbd-user [at] lists
http://lists.linbit.com/mailman/listinfo/drbd-user

vampired at gmail

Aug 9, 2006, 2:48 PM

Post #3 of 3 (547 views)
Permalink

Re: drbd on top of hardware raid [In reply to]

I personally would recommend against RAID5 unless in the most dire
circumstance. Raid 5 performance, especially software raid 5, is brutal.

If it all possible, I recommend raid 10/0+1 or just simply raid 1.

On 8/9/06, Lars Ellenberg <Lars.Ellenberg [at] linbit> wrote:
>
> / 2006-08-09 13:53:39 -0600
> \ Monty Taylor:
> > No, it's an internal LSI MegaRAID configured in Raid5. (not my choice)
> >
> > On the positive side, though, it supports 90M/sec of reading.
>
> if it is an option, try to put it into jbod,
> then do software raid5 on those.
> should help if that controller is "CPU" bound,
> and your file server is not...
>
> and, btw, you probably know that already ...
> raid controllers tend to pretend against the kernel that data was
> on stable storage, even when they indeed only have it still in their
> write back cache.
> if that cache is battery backed, ok.
> if not, and you get data corruption after blackout/brownout,
> don't blame linux, nor the device/file system drivers,
> nor drbd either...
>
> unfortunately sometimes you can not put it in "write through" mode
> (use that cache, but only say its done when its really done), but
> only switch off the cache completely.
> and then your write throughput gets really bad...
>
> so you better make sure you run on some ups, and monitor it,
> and power down the box cleanly in case the ups capacity runs out.
>
> --
> : Lars Ellenberg Tel +43-1-8178292-0 :
> : LINBIT Information Technologies GmbH Fax +43-1-8178292-82 :
> : Schoenbrunner Str. 244, A-1120 Vienna/Europe http://www.linbit.com :
> __
> please use the "List-Reply" function of your email client.
> _______________________________________________
> drbd-user mailing list
> drbd-user [at] lists
> http://lists.linbit.com/mailman/listinfo/drbd-user
>

--
"Do the actors on Unsolved Mysteries ever get arrested because they look
just like the criminal they are playing?"

Christopher

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads