Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: MythTV: Users

Reassemble RAID with mdadm

 

 

First page Previous page 1 2 Next page Last page  View All MythTV users RSS feed   Index | Next | Previous | View Threaded


phillip.p.barnett at gmail

Mar 16, 2010, 3:02 AM

Post #1 of 35 (2071 views)
Permalink
Reassemble RAID with mdadm

I have three disks, sda, sdb, sdc. The OS is on sda, and sdb and sdc
were RAIDed as RAID1, with all my media.

Recently my myth installation got totally screwed up, and I decided to
do a complete upgrade/reinstall. However, this has now broken the
array as all the config material was on the sda drive (d'oh!), and I
need to know how to reassemble the disks. Using mdadm --create didn't
work, as it warned me that the disks were already used for an array.
Should I use assemble, create or build in this instance? The disks
themselves are intact and can be mounted and read individually, but I
do need to reassemble them into a RAID without losing the media
already on them.

Thanks in advance.

Phillip
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


george_mythusers at mari1938

Mar 16, 2010, 5:32 AM

Post #2 of 35 (1999 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On 03/16/2010 05:02 AM, Phillip Barnett wrote:
> I have three disks, sda, sdb, sdc. The OS is on sda, and sdb and sdc
> were RAIDed as RAID1, with all my media.
>
> Recently my myth installation got totally screwed up, and I decided to
> do a complete upgrade/reinstall. However, this has now broken the array
> as all the config material was on the sda drive (d'oh!), and I need to
> know how to reassemble the disks. Using mdadm --create didn't work, as
> it warned me that the disks were already used for an array. Should I use
> assemble, create or build in this instance? The disks themselves are
> intact and can be mounted and read individually, but I do need to
> reassemble them into a RAID without losing the media already on them.
>

What is the output of cat /proc/mdstat?

If your sdb and sdc drives are listed in there as part of an array, all
you would have to do is mount the array to the mountpoint you desire.

If they aren't part of an array, you should just be able to use the
mdadm --assemble command to asseble the two drives into an array.

However, if they are not auto-assembling after a reboot, then most
likely this assembly will only work temprorarily. You may need to look
into your /etc/mdadm.conf file to see how the arrary was identified
previously, and figure out why it's no longer being identified. I've
had the UUID of the array change as part of a system upgrade before,
which was annoying, but fixable.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


phillip.p.barnett at gmail

Mar 16, 2010, 6:00 AM

Post #3 of 35 (2004 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

Thanks George - mmm, I feel pretty stupid now, as I appear to have
fixed it without realising (drunken late-night tinkering, or helpful
tech-savvy fairies?)
Anyway, /proc/mdstat says

$ cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5]
[raid4] [raid10]
md0 : active raid1 sdb1[0] sdc1[1]
976759936 blocks [2/2] [UU]

which to me means it's fixed, and it has auto-assembled post-reboot,
so I just need to mount the device. Hurrah!

Thanks again, sorry for bothering you...

Phillip


On 16 Mar 2010, at 12:32, George Mari wrote:

> On 03/16/2010 05:02 AM, Phillip Barnett wrote:
>> I have three disks, sda, sdb, sdc. The OS is on sda, and sdb and sdc
>> were RAIDed as RAID1, with all my media.
>>
>> Recently my myth installation got totally screwed up, and I decided
>> to
>> do a complete upgrade/reinstall. However, this has now broken the
>> array
>> as all the config material was on the sda drive (d'oh!), and I need
>> to
>> know how to reassemble the disks. Using mdadm --create didn't work,
>> as
>> it warned me that the disks were already used for an array. Should
>> I use
>> assemble, create or build in this instance? The disks themselves are
>> intact and can be mounted and read individually, but I do need to
>> reassemble them into a RAID without losing the media already on them.
>>
>
> What is the output of cat /proc/mdstat?
>
> If your sdb and sdc drives are listed in there as part of an array,
> all you would have to do is mount the array to the mountpoint you
> desire.
>
> If they aren't part of an array, you should just be able to use the
> mdadm --assemble command to asseble the two drives into an array.
>
> However, if they are not auto-assembling after a reboot, then most
> likely this assembly will only work temprorarily. You may need to
> look into your /etc/mdadm.conf file to see how the arrary was
> identified previously, and figure out why it's no longer being
> identified. I've had the UUID of the array change as part of a
> system upgrade before, which was annoying, but fixable.
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


brent.bolin at gmail

Mar 16, 2010, 6:02 AM

Post #4 of 35 (2003 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Tue, Mar 16, 2010 at 5:02 AM, Phillip Barnett
<phillip.p.barnett [at] gmail> wrote:
> I have three disks, sda, sdb, sdc. The OS is on sda, and sdb and sdc were
> RAIDed as RAID1, with all my media.
>
> Recently my myth installation got totally screwed up, and I decided to do a
> complete upgrade/reinstall. However, this has now broken the array as all
> the config material was on the sda drive (d'oh!), and I need to know how to
> reassemble the disks. Using mdadm --create didn't work, as it warned me that
> the disks were already used for an array. Should I use assemble, create or
> build in this instance? The disks themselves are intact and can be mounted
> and read individually, but I do need to reassemble them into a RAID without
> losing the media already on them.
>
> Thanks in advance.
>
> Phillip
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>

This is exactly how my system is setup.

sda contains o/s and recorded TV. All are 1TB drives.

sdb and sdc are raid one with videos, image library and music

I backup the entire /etc directory and mysql for the purpose of reconstruction.

Here is my /etc/mdadm/mdadm.conf -

DEVICE /dev/sdb1 /dev/sdc1
ARRAY /dev/md0 level=raid1 devices=/dev/sdb1,/dev/sdc1

And here is /etc/fstab -

/dev/md0 /disk1 ext4 defaults 0 2

Don't know much about rebuilding mdadm raid1 because I have never done so.

Hope this helps.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


stevehodge at gmail

Mar 16, 2010, 6:24 AM

Post #5 of 35 (1991 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Wed, Mar 17, 2010 at 2:02 AM, Brent Bolin <brent.bolin [at] gmail> wrote:

> Don't know much about rebuilding mdadm raid1 because I have never done so.
>

Rebuilding a raid1 is exactly the same as creating it in the first place
with a disk that has data you want to keep. If you've still got the array
(i.e. one disk is still there and the array is degraded) just add a spare
and let it replicate. Otherwise create a degraded array out of the disk with
the data you want to keep (specify the mirror as missing) then add the
spare.

Cheers,
Steve


brent.bolin at gmail

Mar 16, 2010, 7:11 AM

Post #6 of 35 (1989 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Tue, Mar 16, 2010 at 8:24 AM, Steve Hodge <stevehodge [at] gmail> wrote:
> On Wed, Mar 17, 2010 at 2:02 AM, Brent Bolin <brent.bolin [at] gmail> wrote:
>>
>> Don't know much about rebuilding mdadm raid1 because I have never done so.
>
> Rebuilding a raid1 is exactly the same as creating it in the first place
> with a disk that has data you want to keep.  If you've still got the array
> (i.e. one disk is still there and the array is degraded) just add a spare
> and let it replicate. Otherwise create a degraded array out of the disk with
> the data you want to keep (specify the mirror as missing) then add the
> spare.
>
> Cheers,
> Steve
>
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
>

Steve,

Here is the current status of my raid -

mdadm -D /dev/md0
/dev/md0:
Version : 00.90
Creation Time : Mon Dec 14 19:17:08 2009
Raid Level : raid1
Array Size : 976759936 (931.51 GiB 1000.20 GB)
Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Tue Mar 16 09:00:38 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

UUID : d376cfd0:a438be3c:5f1e94b5:d63edfe3 (local to host myth)
Events : 0.283797

Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 33 1 active sync /dev/sdc1

So if the status ever changes to degraded(not sure what it would
change to if bad sectors etc...) could I simply bring the box down and
replace with another drive? I don't have any spares defined trying to
keep power usage as low as possible.

Or is there an added set of commands to run when the bad drive is replaced?
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


drescherjm at gmail

Mar 16, 2010, 7:58 AM

Post #7 of 35 (1992 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

> Here is the current status of my raid -
>
> mdadm -D /dev/md0
> /dev/md0:
>        Version : 00.90
>  Creation Time : Mon Dec 14 19:17:08 2009
>     Raid Level : raid1
>     Array Size : 976759936 (931.51 GiB 1000.20 GB)
>  Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
>   Raid Devices : 2
>  Total Devices : 2
> Preferred Minor : 0
>    Persistence : Superblock is persistent
>
>    Update Time : Tue Mar 16 09:00:38 2010
>          State : clean
>  Active Devices : 2
> Working Devices : 2
>  Failed Devices : 0
>  Spare Devices : 0
>
>           UUID : d376cfd0:a438be3c:5f1e94b5:d63edfe3 (local to host myth)
>         Events : 0.283797
>
>    Number   Major   Minor   RaidDevice State
>       0       8       17        0      active sync   /dev/sdb1
>       1       8       33        1      active sync   /dev/sdc1
>
> So if the status ever changes to degraded(not sure what it would
> change to if bad sectors etc...) could I simply bring the box down and
> replace with another drive?  I don't have any spares defined trying to
> keep power usage as low as possible.
>
> Or is there an added set of commands to run when the bad drive is replaced?


mdadm /dev/md0 --add /dev/newdrive

John
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


myrdhn at gmail

Mar 16, 2010, 10:03 AM

Post #8 of 35 (1981 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

Steve,

Here is the current status of my raid -

mdadm -D /dev/md0
/dev/md0:
Version : 00.90
Creation Time : Mon Dec 14 19:17:08 2009
Raid Level : raid1
Array Size : 976759936 (931.51 GiB 1000.20 GB)
Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Tue Mar 16 09:00:38 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

UUID : d376cfd0:a438be3c:5f1e94b5:d63edfe3 (local to host myth)
Events : 0.283797

Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 33 1 active sync /dev/sdc1

So if the status ever changes to degraded(not sure what it would
change to if bad sectors etc...) could I simply bring the box down and
replace with another drive? I don't have any spares defined trying to
keep power usage as low as possible.

Or is there an added set of commands to run when the bad drive is replaced?
_______________________________________________

Mirroring is fine for fault tolerance, but you get no speed boost from it.
I would recommend getting 2 more drives if you can afford it and setup a
RAID5 configuration.
Not only do you get a fault tolerant array, can lose 1 drive and not lose
data, but you get a speed boost because the system writes data across all
the drives.

MarcT


_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


brent.bolin at gmail

Mar 16, 2010, 12:04 PM

Post #9 of 35 (1971 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Tue, Mar 16, 2010 at 12:03 PM, MarcT <myrdhn [at] gmail> wrote:
> Steve,
>
> Here is the current status of my raid -
>
> mdadm -D /dev/md0
> /dev/md0:
>        Version : 00.90
>  Creation Time : Mon Dec 14 19:17:08 2009
>     Raid Level : raid1
>     Array Size : 976759936 (931.51 GiB 1000.20 GB)
>  Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
>   Raid Devices : 2
>  Total Devices : 2
> Preferred Minor : 0
>    Persistence : Superblock is persistent
>
>    Update Time : Tue Mar 16 09:00:38 2010
>          State : clean
>  Active Devices : 2
> Working Devices : 2
>  Failed Devices : 0
>  Spare Devices : 0
>
>           UUID : d376cfd0:a438be3c:5f1e94b5:d63edfe3 (local to host myth)
>         Events : 0.283797
>
>    Number   Major   Minor   RaidDevice State
>       0       8       17        0      active sync   /dev/sdb1
>       1       8       33        1      active sync   /dev/sdc1
>
> So if the status ever changes to degraded(not sure what it would
> change to if bad sectors etc...) could I simply bring the box down and
> replace with another drive?  I don't have any spares defined trying to
> keep power usage as low as possible.
>
> Or is there an added set of commands to run when the bad drive is replaced?
> _______________________________________________
>
> Mirroring is fine for fault tolerance, but you get no speed boost from it.
> I would recommend getting 2 more drives if you can afford it and setup a
> RAID5 configuration.
> Not only do you get a fault tolerant array, can lose 1 drive and not lose
> data, but you get a speed boost because the system writes data across all
> the drives.
>
> MarcT
>
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>

Believe you can do a raid5 configuration with 3 drives. However
generating the parity bit slows things down on the write.

With 4 drives you would be better using RAID 0+1

Currently see no bottleneck using raid1. That's recording 4 HD
channels at a time. Flagging set for medium using 2 instances. And
often times running a single Handbrake rip.

Never really did get an answer to my question. But from a little
googling looks like a failing drive would need to be removed. And
then a replacement drive added.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


brent.bolin at gmail

Mar 16, 2010, 12:05 PM

Post #10 of 35 (1974 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Tue, Mar 16, 2010 at 2:04 PM, Brent Bolin <brent.bolin [at] gmail> wrote:
> On Tue, Mar 16, 2010 at 12:03 PM, MarcT <myrdhn [at] gmail> wrote:
>> Steve,
>>
>> Here is the current status of my raid -
>>
>> mdadm -D /dev/md0
>> /dev/md0:
>>        Version : 00.90
>>  Creation Time : Mon Dec 14 19:17:08 2009
>>     Raid Level : raid1
>>     Array Size : 976759936 (931.51 GiB 1000.20 GB)
>>  Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
>>   Raid Devices : 2
>>  Total Devices : 2
>> Preferred Minor : 0
>>    Persistence : Superblock is persistent
>>
>>    Update Time : Tue Mar 16 09:00:38 2010
>>          State : clean
>>  Active Devices : 2
>> Working Devices : 2
>>  Failed Devices : 0
>>  Spare Devices : 0
>>
>>           UUID : d376cfd0:a438be3c:5f1e94b5:d63edfe3 (local to host myth)
>>         Events : 0.283797
>>
>>    Number   Major   Minor   RaidDevice State
>>       0       8       17        0      active sync   /dev/sdb1
>>       1       8       33        1      active sync   /dev/sdc1
>>
>> So if the status ever changes to degraded(not sure what it would
>> change to if bad sectors etc...) could I simply bring the box down and
>> replace with another drive?  I don't have any spares defined trying to
>> keep power usage as low as possible.
>>
>> Or is there an added set of commands to run when the bad drive is replaced?
>> _______________________________________________
>>
>> Mirroring is fine for fault tolerance, but you get no speed boost from it.
>> I would recommend getting 2 more drives if you can afford it and setup a
>> RAID5 configuration.
>> Not only do you get a fault tolerant array, can lose 1 drive and not lose
>> data, but you get a speed boost because the system writes data across all
>> the drives.
>>
>> MarcT
>>
>>
>> _______________________________________________
>> mythtv-users mailing list
>> mythtv-users [at] mythtv
>> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>>
>
> Believe you can do a raid5 configuration with 3 drives.  However
> generating the parity bit slows things down on the write.
>
> With 4 drives you would be better using RAID 0+1
>
> Currently see no bottleneck using raid1.  That's recording 4 HD
> channels at a time.  Flagging set for medium using 2 instances.  And
> often times running a single Handbrake rip.
>
> Never really did get an answer to my question.  But from a little
> googling looks like a failing drive would need to be removed.  And
> then a replacement drive added.
>

I would add to this that I have Myth logging set to /dev/null
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


drescherjm at gmail

Mar 16, 2010, 12:11 PM

Post #11 of 35 (1975 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

> Believe you can do a raid5 configuration with 3 drives.  However
> generating the parity bit slows things down on the write.
>

That depends on tuning and how often the writes occur. The forced
flushes with multiple streams work very much against performance.
Tuning the stripe cache can reduce this problem but that risks
corruption in the unlikely event that the system looses power or
crashes.

John
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


gmitch at woodlea

Mar 16, 2010, 3:45 PM

Post #12 of 35 (1951 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

> With 4 drives you would be better using RAID 0+1


Probably better off using RAID 1+0 (a stripe of mirrors), rather than 0+1 (a
mirror of a stripe)


G


_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


stevehodge at gmail

Mar 16, 2010, 7:57 PM

Post #13 of 35 (1914 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Wed, Mar 17, 2010 at 11:45 AM, Graham Mitchell <gmitch [at] woodlea>wrote:

> > With 4 drives you would be better using RAID 0+1
>
>
> Probably better off using RAID 1+0 (a stripe of mirrors), rather than 0+1
> (a
> mirror of a stripe)
>

Yes. Just to elaborate, the difference is in what happens when you lose a
drive. With raid 0+1 if you lose a drive you lose the entire stripe - you're
effectively back to a raid 0. With raid 1+0 if you lose a drive you've got
one degraded mirrored pair but the rest of the mirrors in the stripe are
still at full performance. However it's possible that mdadm is actually
smart enough to handle these two cases identically anyway (but I'm not 100%
sure).

With mdadm you'd be better off using raid10 which manages striping and
mirroring together at the block level. It's both more flexible and
potentially better performing. Basically it spreads X copies of the data
over Y drives with the copies staggered physically. Look it up on wikipedia
for more details.

Cheers,
Steve


myrdhn at gmail

Mar 17, 2010, 4:04 PM

Post #14 of 35 (1808 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

> Believe you can do a raid5 configuration with 3 drives. However
> generating the parity bit slows things down on the write.
>
> With 4 drives you would be better using RAID 0+1

RAID 5 requires a mimimum of 3 yes, but at that point it's not significantly
different than a RAID 3 setup.

IMO RAID 0+1 is the dumbest use of available HDD's. Yes you are fault
tolerant but if you lose 1 drive from each stripe you are hosed. RAID 10
would be a little better as you can lose 1 drive from each mirror but if you
lose both from the same mirror you are in the same boat.

I have not seen a performance hit in my system running RAID 5. You get some
of the benefits of a stripe with the fault tolerance of being able to lose 1
drive, you also don't lose the storage from the extra drive. For example, in
a RAID 10 or 0+1 you are using 4 drives but your total storage is no more
than having 2 drives striped, with fault tolerance. RAID 5 gives you the
storage space of all 4 drive minus the space needed for parity, essentially
4-1, so your total storage space is about that of a 3 drive stripe with the
ability to lose one drive and not lose the data.

RAID 6 would be the best setup (4-2, similar to the 4 drive 10 or 0+1 setup
but you can lose any 2 drives ant not 1 drive from each mirror/stripe)
However, for software RAID, I don't think we are ready for that yet with
today's systems and you would see the performance hit that was commented on
above. You would need a hardware RAID controller to make use of RAID 6
without seeing a significant hit to performance.

My RAID 5 drive is only used for media storage, I have a raid mirror setup
for the / and /boot partitions.
As of this time I have been able to capture 6 concurrent HD recordings, 2
per HD tuner, without incident.

MarcT

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


freedenizen at gmail

Mar 17, 2010, 5:54 PM

Post #15 of 35 (1792 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Wed, Mar 17, 2010 at 4:04 PM, MarcT <myrdhn [at] gmail> wrote:
> RAID 6 would be the best setup (4-2, similar to the 4 drive 10 or 0+1 setup
> but you can lose any 2 drives ant not 1 drive from each mirror/stripe)
> However, for software RAID, I don't think we are ready for that yet with
> today's systems and you would see the performance hit that was commented on
> above. You would need a hardware RAID controller to make use of RAID 6
> without seeing a significant hit to performance.

I'm not so sure about this, I mean we are talking about TV recordings,
even HD, which is pretty light IO wise. I've seen quite good software
RAID 6 performance, and other people have as well:
http://louwrentius.blogspot.com/2009/06/linux-raid-6-performance-using-software.html

Even without making a massive array like that, your performance with
RAID6 will be more then good enough for your recording needs, and
being able to survive 2 disks failures is very nice, especially as you
end up increasing the size of the array (which happens to all of us).

One thing to note, mdadm 3.1 and kernel => 2.6.32 should support RAID
level modification, meaning you can move from one raid level to
another, it also support other online changes. I wouldn't go for 3.1
yet myself, as it is still pretty fresh, and I'm sure there are bugs
to work it. Also to note those features require superblock ver 1.1
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


k at adamski

Mar 18, 2010, 6:45 AM

Post #16 of 35 (1713 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Wed, 2010-03-17 at 19:04 -0400, MarcT wrote:

> My RAID 5 drive is only used for media storage, I have a raid mirror setup
> for the / and /boot partitions.

I wanted to add to this, like you I have /boot as RAID1, it uses the
same drives that are part of the RAID6 for the media files.
My /boot RAID1 is a mirror of 8 drives, not 2 mirror and 6 spares, but 8
drives in a mirror:
md0 : active raid1 sdc1[0] sdf1[7] sdb1[6] sdd1[5] sde1[4] sdg1[3]
sda1[2] sdh1[1]
240832 blocks [8/8] [UUUUUUUU]


_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


drescherjm at gmail

Mar 18, 2010, 7:55 AM

Post #17 of 35 (1714 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Thu, Mar 18, 2010 at 9:45 AM, Krzysztof Adamski <k [at] adamski> wrote:
> On Wed, 2010-03-17 at 19:04 -0400, MarcT wrote:
>
>> My RAID 5 drive is only used for media storage, I have a raid mirror setup
>> for the / and /boot partitions.
>
> I wanted to add to this, like you I have /boot as RAID1, it uses the
> same drives that are part of the RAID6 for the media files.
> My /boot RAID1 is a mirror of 8 drives, not 2 mirror and 6 spares, but 8
> drives in a mirror:
> md0 : active raid1 sdc1[0] sdf1[7] sdb1[6] sdd1[5] sde1[4] sdg1[3]
> sda1[2] sdh1[1]
>      240832 blocks [8/8] [UUUUUUUU]
>

I do that on all linux raid arrays here at work.

Here is one:
md0 : active raid1 sdh1[6](S) sdg1[5] sdf1[7](S) sde1[4] sdd1[3]
sdc1[2] sdb1[1] sda1[0]
256896 blocks [6/6] [UUUUUU]

There are 2 spares here because I just added a hard drive to expand
the image data storage lvm.

md3 : active raid5 sdh6[6] sdg6[5] sdf6[7](S) sde6[4] sdd6[3] sdc6[2]
sdb6[1] sda6[0]
3540604800 blocks super 0.91 level 5, 64k chunk, algorithm 2
[7/7] [UUUUUUU]
[>....................] reshape = 2.0% (14465152/708120960)
finish=687.2min speed=16819K/sec

This is a slower dual core so the reshape will take some time..

John
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


k at adamski

Mar 18, 2010, 10:28 AM

Post #18 of 35 (1699 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Thu, 2010-03-18 at 10:55 -0400, John Drescher wrote:
> On Thu, Mar 18, 2010 at 9:45 AM, Krzysztof Adamski <k [at] adamski> wrote:
> > On Wed, 2010-03-17 at 19:04 -0400, MarcT wrote:
> >
> >> My RAID 5 drive is only used for media storage, I have a raid mirror setup
> >> for the / and /boot partitions.
> >
> > I wanted to add to this, like you I have /boot as RAID1, it uses the
> > same drives that are part of the RAID6 for the media files.
> > My /boot RAID1 is a mirror of 8 drives, not 2 mirror and 6 spares, but 8
> > drives in a mirror:
> > md0 : active raid1 sdc1[0] sdf1[7] sdb1[6] sdd1[5] sde1[4] sdg1[3]
> > sda1[2] sdh1[1]
> > 240832 blocks [8/8] [UUUUUUUU]
> >
>
> I do that on all linux raid arrays here at work.
>
> Here is one:
> md0 : active raid1 sdh1[6](S) sdg1[5] sdf1[7](S) sde1[4] sdd1[3]
> sdc1[2] sdb1[1] sda1[0]
> 256896 blocks [6/6] [UUUUUU]
>
> There are 2 spares here because I just added a hard drive to expand
> the image data storage lvm.
>
> md3 : active raid5 sdh6[6] sdg6[5] sdf6[7](S) sde6[4] sdd6[3] sdc6[2]
> sdb6[1] sda6[0]
> 3540604800 blocks super 0.91 level 5, 64k chunk, algorithm 2
> [7/7] [UUUUUUU]
> [>....................] reshape = 2.0% (14465152/708120960)
> finish=687.2min speed=16819K/sec
>
> This is a slower dual core so the reshape will take some time..

I get faster rebuilds with a Pentium D, check you max and min in
/proc/sys/dev/raid/speed_limit_max are higher then defaults.

K

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


drescherjm at gmail

Mar 18, 2010, 10:41 AM

Post #19 of 35 (1696 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

> I get faster rebuilds with a Pentium D, check you max and min in
> /proc/sys/dev/raid/speed_limit_max are higher then defaults.
>
I already did that. This is a reshape which takes much longer than a
normal rebuild since all data needs to move. I believe on the same
machine a rebuild is 4 to 5 times faster.

John
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


paul10 at planar

Mar 19, 2010, 1:16 AM

Post #20 of 35 (1630 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On RAID6, RAID5, RAID0+1 and others.

On a modern CPU, there is a theoretical slowdown calculating parity, but
my quad core runs about 5% of one core when I'm thrashing the drives hard.
Frankly, any modern system has CPU to burn, and I don't believe parity
calcs are a real-life impact on performance.

RAID5 is the most space/cost efficient. You get n-1 space, so with a 5
drive array, you get 4 disks worth of space.

I had a lot of problems with RAID5 and consumer grade disks. My disks are
generally failing about 1 every 3-6 months, in a 6 disk array. Which makes
sense if you figure the life of a drive is 2-3 years. My theory here is
that consumer grade drives are made to a budget. They give a 3 year
warranty, but they're assuming you don't run it 24x7. My myth setup runs
24x7 and is usually reading or writing - which means the drives never
really idle out. I think someone (in all the various drive manufacturers)
has figured it is cheaper to replace drives under warranty for the handful
of mythtv users who run 24x7 than it is to build a good drive for the 99%
of users who turn their computer on once a day.

My problem is that if I lose a drive once every 3-6 months, and it takes
me a couple of weeks to get around to a replacement, then there is a real
chance of losing another disk in that timeframe. Actually, if you're a
klutz like me, more likely is you'll lose one drive, decide that, really,
my machine is hot pluggable (it is, honest), pull out the dead drive to put
in a new one, and accidentally knock out the cable for one of the other
drives. Broken array.

I thought about going RAID5 with a spare, but my understanding was that if
you lose a drive, it immediately goes into rebuild. Which entails reading
every sector on every drive, and that can trigger another drive to fail -
result is broken array again. With RAID6, when one drive fails what you
have left is essentially RAID5. When you put a new drive in, it rebuilds,
but the odds of two drives failing during that rebuild are lower - so
safer.

Anyway, long story short, I'm running RAID6 with 6x1TB, giving 4TB
useable. With ext4, the performance is sufficient that I'm more than
capable of servicing 3xtuners at HD quality, plus commercial flagging, plus
playback on 2 frontends. In theory other configurations might be faster,
but this is fast enough, low maintenance, relatively cheap, and easy to
configure.

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


drescherjm at gmail

Mar 19, 2010, 5:40 AM

Post #21 of 35 (1615 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

> On a modern CPU, there is a theoretical slowdown calculating parity, but
> my quad core runs about 5% of one core when I'm thrashing the drives hard.
> Frankly, any modern system has CPU to burn, and I don't believe parity
> calcs are a real-life impact on performance.
>

Agreed. Even on 5 year old systems the cput hits 7% max unless the you
are rebuilding or reshaping then it can hit 25% on 1 core..

> RAID5 is the most space/cost efficient.  You get n-1 space, so with a 5
> drive array, you get 4 disks worth of space.
>
Agreed again

>
> I had a lot of problems with RAID5 and consumer grade disks.  My disks are
> generally failing about 1 every 3-6 months, in a 6 disk array.  Which makes
> sense if you figure the life of a drive is 2-3 years.  My theory here is
> that consumer grade drives are made to a budget.  They give a 3 year
> warranty, but they're assuming you don't run it 24x7.

There is something very strange here. I have 100+ such disks in raid
and I see 1 to 3 total failures per year running 24/7/365. And these
are systems that I guarantee are used more than your Mythtv box.

John
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


phillip.p.barnett at gmail

Mar 19, 2010, 6:43 AM

Post #22 of 35 (1590 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

I've got 320 disks in an Avid ISIS (not strictly RAID, but obscure
algorithmic data spreading) array running 24/7, and lost about 3 in
the past year, so about 1% a year.
Granted these disks are probably not consumer grade, but I put 4
Maxtor disks in two Tivos in 2001, and they're still running, and
actually recording every second of every day since.

Anecdata, eh!

Phillip

Sent from my iPhone

On 19 Mar 2010, at 12:40, John Drescher <drescherjm [at] gmail> wrote:

>> On a modern CPU, there is a theoretical slowdown calculating
>> parity, but
>> my quad core runs about 5% of one core when I'm thrashing the
>> drives hard.
>> Frankly, any modern system has CPU to burn, and I don't believe
>> parity
>> calcs are a real-life impact on performance.
>>
>
> Agreed. Even on 5 year old systems the cput hits 7% max unless the you
> are rebuilding or reshaping then it can hit 25% on 1 core..
>
>> RAID5 is the most space/cost efficient. You get n-1 space, so with
>> a 5
>> drive array, you get 4 disks worth of space.
>>
> Agreed again
>
>>
>> I had a lot of problems with RAID5 and consumer grade disks. My
>> disks are
>> generally failing about 1 every 3-6 months, in a 6 disk array.
>> Which makes
>> sense if you figure the life of a drive is 2-3 years. My theory
>> here is
>> that consumer grade drives are made to a budget. They give a 3 year
>> warranty, but they're assuming you don't run it 24x7.
>
> There is something very strange here. I have 100+ such disks in raid
> and I see 1 to 3 total failures per year running 24/7/365. And these
> are systems that I guarantee are used more than your Mythtv box.
>
> John
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


freedenizen at gmail

Mar 19, 2010, 7:00 AM

Post #23 of 35 (1594 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Fri, Mar 19, 2010 at 5:40 AM, John Drescher <drescherjm [at] gmail> wrote:
> There is something very strange here. I have 100+ such disks in raid
> and  I see 1 to 3 total failures per year running 24/7/365. And these
> are systems that I guarantee are used more than your Mythtv box.
>
> John

Seems about right to me, I see 3-4 drive failures a year that I
replace under warranty on a 6 disk array. This was true when I was
using 500GB seagate drives, through the 1.5TB WD drives I have in
there now, and the other ones I had in between. I'd run enterprise
class drives, but they don't make them in large capacities, and I
don't want to have to double the number of disks for the same space.
I'm also running raid6 after dealing with the a drive failure during
an array rebuild using raid5, and yes, I do scrub my arrays once a
month as well to check for errors.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


drescherjm at gmail

Mar 19, 2010, 7:39 AM

Post #24 of 35 (1588 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Fri, Mar 19, 2010 at 10:00 AM, freedenizen <freedenizen [at] gmail> wrote:
> On Fri, Mar 19, 2010 at 5:40 AM, John Drescher <drescherjm [at] gmail> wrote:
>> There is something very strange here. I have 100+ such disks in raid
>> and  I see 1 to 3 total failures per year running 24/7/365. And these
>> are systems that I guarantee are used more than your Mythtv box.
>>
>> John
>
> Seems about right to me, I see 3-4 drive failures a year that I
> replace under warranty on a 6 disk array.  This was true when I was
> using 500GB seagate drives, through the 1.5TB WD drives I have in
> there now, and the other ones I had in between.  I'd run enterprise
> class drives, but they don't make them in large capacities, and I
> don't want to have to double the number of disks for the same space.
> I'm also running raid6 after dealing with the a drive failure during
> an array rebuild using raid5, and yes, I do scrub my arrays once a
> month as well to check for errors.
>
I am using desktop WDC and Seagate drives (even 7200.11 drives) and do
not experience any where near this failure rate. To me this is insane.

John
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


drescherjm at gmail

Mar 19, 2010, 7:41 AM

Post #25 of 35 (1590 views)
Permalink
Re: Reassemble RAID with mdadm [In reply to]

On Fri, Mar 19, 2010 at 10:39 AM, John Drescher <drescherjm [at] gmail> wrote:
> On Fri, Mar 19, 2010 at 10:00 AM, freedenizen <freedenizen [at] gmail> wrote:
>> On Fri, Mar 19, 2010 at 5:40 AM, John Drescher <drescherjm [at] gmail> wrote:
>>> There is something very strange here. I have 100+ such disks in raid
>>> and  I see 1 to 3 total failures per year running 24/7/365. And these
>>> are systems that I guarantee are used more than your Mythtv box.
>>>
>>> John
>>
>> Seems about right to me, I see 3-4 drive failures a year that I
>> replace under warranty on a 6 disk array.  This was true when I was
>> using 500GB seagate drives, through the 1.5TB WD drives I have in
>> there now, and the other ones I had in between.  I'd run enterprise
>> class drives, but they don't make them in large capacities, and I
>> don't want to have to double the number of disks for the same space.
>> I'm also running raid6 after dealing with the a drive failure during
>> an array rebuild using raid5, and yes, I do scrub my arrays once a
>> month as well to check for errors.
>>
> I am using desktop WDC and Seagate drives (even 7200.11 drives) and do
> not experience any where near this failure rate. To me this is insane.
>

Have you guys checked your power supplies? System overheating?

John
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users

First page Previous page 1 2 Next page Last page  View All MythTV users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.