Mailing List Archive: MythTV: Mythtvnz

Myth & Raid. SW or HW..

Index | Next | Previous | View Threaded

johanschuld at gmail

Jul 18, 2012, 1:42 AM

Post #1 of 20 (951 views)
Permalink

Myth & Raid. SW or HW..

Hi All,

A while back the HD on my mythbox died and had to replace it with an old
250Gig ATA drive I had lying around. Not the best option with HD recordings
etc.

So, as HD will inevitably die, my plan is to buy some new HD's and put them
in a RAID array. Thinking RAID1 as I currently don't have the funds to buy
more than 2 drives.

I've done quite some research so far about the options and wondered what
the easiest and most reliable solution. I.e. I know people having a HW Raid
using a Dell Perc 5/i card. These are so common they can be picked up for
less than $100NZ inc shipping from ebay and with some SAS to SATA cables
they run fine with SATA drives. A cheaper option would obviously be SW
Raid, but not that much cheaper.

Regardless of SW or HW, I'm thinking of 2x 2TB drives, in Raid1. I've been
looking at the WD Green ones as they are competitively priced and silent
(pc is under the TV). That drive has been discussed many times here so I am
aware that they might have issues in SW raid, I'm just still looking at a
similar drive from another vendor.

As Raid1 is really not that much overhead, should I just not bother with HW
raid, and let linux do the job? (Opensuse 12.1) . Can I transfer a already
existing non-raid install to a SW raid1 config? For now I'll run the OS and
media storage on one raid volume, I might split that out later when I can
afford some more drives.

I can find quite some info about Raid, but not so much in combination with
Myth.. don't want to do stuff that's overkill, but like to have stuff that
works (particulary when one drive fails ;)

Cheers
J.

james at booths

Jul 18, 2012, 1:55 AM

Post #2 of 20 (943 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On Wed, 18 Jul 2012 20:42:27 Johan Schuld wrote:

Hi All,

A while back the HD on my mythbox died and had to replace it with an old
250Gig ATA drive I had lying around. Not the best option with HD recordings
etc.

So, as HD will inevitably die, my plan is to buy some new HD's and put them in
a RAID array. Thinking RAID1 as I currently don't have the funds to buy more
than 2 drives.

I've done quite some research so far about the options and wondered what the
easiest and most reliable solution. I.e. I know people having a HW Raid using
a Dell Perc 5/i card. These are so common they can be picked up for less than
$100NZ inc shipping from ebay and with some SAS to SATA cables they run fine
with SATA drives. A cheaper option would obviously be SW Raid, but not that
much cheaper.

Regardless of SW or HW, I'm thinking of 2x 2TB drives, in Raid1. I've been
looking at the WD Green ones as they are competitively priced and silent (pc
is under the TV). That drive has been discussed many times here so I am aware
that they might have issues in SW raid, I'm just still looking at a similar
drive from another vendor.

As Raid1 is really not that much overhead, should I just not bother with HW
raid, and let linux do the job? (Opensuse 12.1) . Can I transfer a already
existing non-raid install to a SW raid1 config? For now I'll run the OS and
media storage on one raid volume, I might split that out later when I can
afford some more drives.

I can find quite some info about Raid, but not so much in combination with
Myth.. don't want to do stuff that's overkill, but like to have stuff that
works (particulary when one drive fails ;)

Cheers
J.

I run 2 x RAID1 (4 disks total) using software RAID. System disk is separate
again. I recently had my first disk failure, it was a low-level interface
failure and system would not even boot with the disk connected. Just
disconnected the faulty disk and was back into business with the mirror. While
I waited for my warranty replacement disk I did an upgrade from 11.04 to 12.04
and myth 0.25. When the disk arrived I just threw it in, added it to the
array, and away we went. Definitely worth doing RAID, but I should probably
have gone for RAID5 as it would be much easier to add capacity.

hads at nice

Jul 18, 2012, 1:59 AM

Post #3 of 20 (938 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On 18/07/12 20:42, Johan Schuld wrote:
> So, as HD will inevitably die, my plan is to buy some new HD's and put
> them in a RAID array. Thinking RAID1 as I currently don't have the funds
> to buy more than 2 drives.

Why? Do you need minimal downtime of your recordings? That's what RAID
is for.

> Regardless of SW or HW, I'm thinking of 2x 2TB drives, in Raid1. I've
> been looking at the WD Green ones as they are competitively priced and
> silent (pc is under the TV). That drive has been discussed many times
> here so I am aware that they might have issues in SW raid, I'm just
> still looking at a similar drive from another vendor.

The WD Green drives (and other consumer drives) have issues with any
RAID, be it software or hardware. Their firmware just isn't designed for it.

I use WD Green drives, but I don't use RAID, I don't consider the
recordings worth it. I do have backups though.

hads
--
http://nice.net.nz

_______________________________________________
mythtvnz mailing list
mythtvnz [at] lists
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

criggie at criggie

Jul 18, 2012, 2:02 AM

Post #4 of 20 (939 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On 18/07/12 20:42, Johan Schuld wrote:
> A while back the HD on my mythbox died and had to replace it with an old
> 250Gig ATA drive I had lying around. Not the best option with HD
> recordings etc.
>
> So, as HD will inevitably die, my plan is to buy some new HD's and put
> them in a RAID array. Thinking RAID1 as I currently don't have the funds
> to buy more than 2 drives.

All storage will die eventually. Thats why you backup the stuff you
can't replace,

> I've done quite some research so far about the options and wondered what
> the easiest and most reliable solution. I.e. I know people having a HW
> Raid using a Dell Perc 5/i card. These are so common they can be picked
> up for less than $100NZ inc shipping from ebay and with some SAS to SATA
> cables they run fine with SATA drives. A cheaper option would obviously
> be SW Raid, but not that much cheaper.
>
> Regardless of SW or HW, I'm thinking of 2x 2TB drives, in Raid1. I've
> been looking at the WD Green ones as they are competitively priced and
> silent (pc is under the TV). That drive has been discussed many times
> here so I am aware that they might have issues in SW raid, I'm just
> still looking at a similar drive from another vendor.
>
> As Raid1 is really not that much overhead, should I just not bother with
> HW raid, and let linux do the job? (Opensuse 12.1) . Can I transfer a
> already existing non-raid install to a SW raid1 config? For now I'll run
> the OS and media storage on one raid volume, I might split that out
> later when I can afford some more drives.
>
> I can find quite some info about Raid, but not so much in combination
> with Myth.. don't want to do stuff that's overkill, but like to have
> stuff that works (particulary when one drive fails ;)

I've avoided green drives myself. They've never been good fullstop, let
alone with raid on top.

You can raid1 the OS if you want, but don't bother with raid anything
for your TV recordings. You're not trying to save these files forever.
If theres something broadcast that you do want to keep, write it to DVD
or export it to AVI and put it with your important data.

So I'd suggest this:

sda 2TB drive | 100 GB | 1700 GB |
sdb 2TB drive | 100 GB | 1700 GB |

Create a raid1 for OS/mysql/exported videos out of sda1 and sdb1
So root is on /dev/md0 instead of /dev/sdXN

And use the 2x 1700GB as Mythtv's Storage Groups. So its effectively a
raid0 on those two partitions, but if a single drive dies you only loose
half the files not all the files.

Software raid is faster than cheap hardware raid on a modern CPU (that's
anything later than a P4)

Good hardware raid cards cost more than your entire system.
Excellent hardware raid cards cost more than a car.

Adding capacity in the future is a doddle - you add drives, tell myth to
use sdc/sdd... as more storage group locations and its all done.

Just remember, its only TV - not worth overengineering.

--
Criggie

http://criggie.org.nz/

_______________________________________________
mythtvnz mailing list
mythtvnz [at] lists
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

stevehodge at gmail

Jul 18, 2012, 2:19 AM

Post #5 of 20 (952 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On Wed, Jul 18, 2012 at 8:42 PM, Johan Schuld <johanschuld [at] gmail> wrote:

> So, as HD will inevitably die, my plan is to buy some new HD's and put
> them in a RAID array. Thinking RAID1 as I currently don't have the funds to
> buy more than 2 drives.

I assume your goal is data security? RAID is not a substitute for backups
so if you have really critical data on that machine you should be backing
it up.

> I've done quite some research so far about the options and wondered what
> the easiest and most reliable solution. I.e. I know people having a HW Raid
> using a Dell Perc 5/i card. These are so common they can be picked up for
> less than $100NZ inc shipping from ebay and with some SAS to SATA cables
> they run fine with SATA drives. A cheaper option would obviously be SW
> Raid, but not that much cheaper.
>

$100 is about 60% of the price of a 2TB drive. So you can have 2TB of
storage on hardware RAID 1 for about $420 or you can have 4TB of storage on
software RAID 5 for $480. There are still a lot of people who favour
hardware RAID but there really aren't many advantages unless you go high
end. The CPU hit for software RAID is very low for modern CPUs and software
RAID is far more flexible. Keep in mind that hardware RAID involves an
extra device so it actually increases the chance of failures. And if the
controller fails and you're running something like RAID 5 or 6 you might
not be able to recover the array without replacing the controller card with
an identical model. That could be difficult 4 or 5 years down the track.

> As Raid1 is really not that much overhead, should I just not bother with
> HW raid, and let linux do the job? (Opensuse 12.1) . Can I transfer a
> already existing non-raid install to a SW raid1 config? For now I'll run
> the OS and media storage on one raid volume, I might split that out later
> when I can afford some more drives.

Yes, with mdadm you can set up a degraded array. You can also mirror your
existing drive. So you could put in one new 2TB drive and mirror the
existing drive to the new one. Keep them in RAID1 until the old drive
fails. Use the remaining space on the new drive as separate storage group
in Myth. Later you can add another 2TB drive and mirror both the OS part
and the storage group if you like (or you could use the extra space on the
new drive as a 2nd storage group). If you can get 2 drives now you could
run them as a degraded RAID 5, but note that a degraded RAID 5 is
effectively just a RAID 0 (if either drive fails you lose everything). I
wouldn't run that way for any length of time.

My recommendation would be three 2TB drives in software RAID 5. If you
really can't stretch to 3 drives then look at mirroring the existing 250GB
across 2 drives and use the remaining space as separate storage groups.
That'll give you 250GB of mirrored capacity and 3.5TB additional space.
That way you maximise capacity and yet still have a fair bit of protection
from data loss - you might still lose half your recordings but if go with
RAID1 you can only record half as much anyway.

Personally I don't bother with LVM. mdadm can be used to reshape RAID
arrays so you can increase capacity by adding disks later and then simply
use jfs or xfs to extend the filesystem to take advantage of the larger
size. I've done this myself in moving from a 3x1.5TB RAID 5 to a 4x1.5TB
RAID 5.

Cheers,
Steve

millst at np

Jul 18, 2012, 2:20 AM

Post #6 of 20 (939 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

>All storage will die eventually. Thats why you backup the stuff you can't
replace,

I would avoid raid for mythtv, its just not necessary and can result in
file fragmentation.
I run 5 x 130Gb drives mounted as separate storage groups. That means I can
record 5 programs at once and each is written to a different spindle.
This means the files never get fragmented and the performance is out of
this world.
You can pick up an old DELL server on trademe for a couple of hundred bucks
and build the ultimate mythbackend.
I am running VMWare on mine, so I can also experiment with a non production
version of Myth on the same physical box or other virtual appliances.

Also, if I have a HD failure, I would only lose 1/5th of my recordings, not
really the end of the world given anything important can be downloaded
easily.
This configuration gives far superior performance for mythtv than any RAID
setup ever will and provides adequate redundancy.
it also means if anything goes down, I just slot in a new disk, mount it
and I'm away again in 10 minutes, far quicker than rebuilding RAID arrays.
I backup my OS, database and configuration to a USB flash drive and also
put a whole backup of the boot VMDK file on their every night.
so if I lose the OS, I just reinstall mythbuntu (I leave the disk in the CD
tray) and restore from the backup or I can put in a new disk and copy the
VMDK file across (i boot vmware off a USB flash disk) and I'm away again in
15 minutes.

Sometimes keeping it simple works the best.

Toby

stevehodge at gmail

Jul 18, 2012, 11:13 AM

Post #7 of 20 (917 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On Wed, Jul 18, 2012 at 8:55 PM, James Booth <james [at] booths> wrote:

> **
>
> I run 2 x RAID1 (4 disks total) using software RAID. System disk is
> separate again.
>

Any reason why you chose that setup over software RAID10? RAID10 would give
you the same level of redundancy with better performance.

> Definitely worth doing RAID, but I should probably have gone for RAID5 as
> it would be much easier to add capacity.
>

If the drives are all the same size then it wouldn't be that hard to change
over for you. Take a drive out of each RAID1. Use those 2 drives to create
a degraded RAID5. Copy the data from one of the original RAID1 arrays to
the RAID5. Destroy that RAID1 and add the drive to the RAID5. Copy the data
from the other RAID1 (you don't need to wait for the RAID5 to rebuild).
Reshape the RAID5 to add the last drive. You'd be running the risk of
losing half your data to a single drive failure while the RAID5 is degraded.

Switching to RAID5 would give you extra capacity equal to the size of the
smallest disk so if there is a big difference in the size of the drives
then it might not be worth the effort.

Cheers,
Steve

james at booths

Jul 18, 2012, 12:18 PM

Post #8 of 20 (917 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On Wed, 18 Jul 2012 20:59:47 Hadley Rich wrote:
> On 18/07/12 20:42, Johan Schuld wrote:
> > So, as HD will inevitably die, my plan is to buy some new HD's and put
> > them in a RAID array. Thinking RAID1 as I currently don't have the funds
> > to buy more than 2 drives.
>
> Why? Do you need minimal downtime of your recordings? That's what RAID
> is for.
>
> > Regardless of SW or HW, I'm thinking of 2x 2TB drives, in Raid1. I've
> > been looking at the WD Green ones as they are competitively priced and
> > silent (pc is under the TV). That drive has been discussed many times
> > here so I am aware that they might have issues in SW raid, I'm just
> > still looking at a similar drive from another vendor.
>
> The WD Green drives (and other consumer drives) have issues with any
> RAID, be it software or hardware. Their firmware just isn't designed for it.
>
> I use WD Green drives, but I don't use RAID, I don't consider the
> recordings worth it. I do have backups though.
>
> hads

Many seem to take the view that there is little point to RAID since primary
storage is for TV recordings, which I agree are low value. However, in my case
the bulk of my data is not taken up with TV recordings but movies and music,
which I consider very high value since I don't know how many hours of labour I
put in ripping my rather huge DVD and CD collection.

Performance wise there is no need for RAID, and storage groups make expanding
storage easy. So just go RAID if you really don't want to loose whatever media
you have stored on those disks to a disk failure and to maximise uptime for
your system. But as others have noted, RAID will not help against fire or
theft - that's what backups are for.

_______________________________________________
mythtvnz mailing list
mythtvnz [at] lists
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

millst at np

Jul 18, 2012, 1:57 PM

Post #9 of 20 (919 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

>Many seem to take the view that there is little point to RAID since primary
>storage is for TV recordings, which I agree are low value. However, in my
case
>the bulk of my data is not taken up with TV recordings but movies and
music,
>which I consider very high value since I don't know how many hours of
labour I
>put in ripping my rather huge DVD and CD collection.

This is a good point about videos and music, its wise to remember that RAID
is not a backup strategy.
RAID is a method of avoiding downtime and increasing performance, however
in a myth setup, there is a relatively low cost of downtime so RAID has a
lower value.

I take the same view on Videos and Music, however I don't safeguard this
with RAID.
I simply use the same 1Tb USB drive that I use for configuration backups as
my primary storage of Videos and music, but then use RSYNC to take a weekly
copy of this onto another 1Tb USB drive.

At the end of the day, you don't need the performance of RAID for your
Video and Music drive. A USB drive can easily handle playback of multiple
DVD's. The plus side is that it is also portable should you need it to take
it somewhere. Your backups also don't need to be online, once a week is
usually fine for Videos and Music as they do not change frequently.

I'm a big fan of RAID in large storage situations concerning production
databases and mission critical systems, but it has very little value in a
myth setup and simply serves to complicate matters and reduce performance.

The key issue with RAID for mythtv is fragmentation, fragmentation can
offset the increased perofmance you get from having raid.
The file I/O profile of a mythtv setup is quite different from what RAID
was designed and optimized for.
This is why the myth dev team created storage groups.
If you have 4 recordings being recorded to a raid array at the same time,
then the recordings will not be contiguous, this results in a very high
level of fragmentation. If you divide your raid array up into storage
groups to get around this, then if your array goes down you have a 100%
outage or degradation while it is repaired.
This makes no sense when mythtv has a built in method of maximising
performance and redundancy.

If you used the same disks in your raid array to create 1 storage group
from each disk then your performance will be better than any RAID
configuration and your risk of data loss will be lower than every single
raid configuration except RAID 10. You will also have more storage capacity
than any RAID configuration.

You can also take storage groups offline and replace disks while people are
still watching TV (so long as it is not a program that is on the spindle
you are replacing).

Toby

noel at igrin

Jul 18, 2012, 4:02 PM

Post #10 of 20 (916 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

snip..
> The key issue with RAID for mythtv is fragmentation, fragmentation can
> offset the increased perofmance you get from having raid.
> The file I/O profile of a mythtv setup is quite different from what RAID
> was designed and optimized for.
>

My own experience of to raid or not to raid..
I have software raid 1 for my system drive, and a raid 5 array for my
storage (not using storage groups and the drives run via a $50 fake-raid
card bought off TradeMe - I bought 2, so I could have one as a spare) on a
dedicated back end, and watch TV via an old ex-lease compaq. We often
have 6 recordings happening simultaneously plus watching something else.
We can't detect any performance issues & the system has been up for YEARS
(save for some power cuts).
Having suffered dive failure from buying cheap second hand **GASP** drives
in the early days (OK I admit it was more than one drive, my own fault I
know), I simply changed the drive & away we went within half an hour -
hasd to fight my way in to where the server was bueried. no fuss no
fiddle with a simple hot-swap. The **NEW** drive is larger than the
remaining array drives, so I have wasted a bit of space etc on that
drive.. but hey, at the end of the day our music vids & all the kid's
programmes are still there & I don't need to shuffle through a 100 or so
DVDs to find what I want to watch.
With fragmentation, I think it's mind over matter, if you don't mind, then
it doesn't matter - as long as I can't TELL it's there!
I will eventually up-size the remainder of the drives but that won't
happen till there is another failure.

_______________________________________________
mythtvnz mailing list
mythtvnz [at] lists
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

johanschuld at gmail

Jul 18, 2012, 6:46 PM

Post #11 of 20 (912 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

Hi All,

Thanks for all the response. I agree I still need to make regular backups,
which I'm doing. It's just a pain that when it goes down, I have to get new
HW, rebuild it and have down time while doing that which does not boast
well wit my better half. That's why I'm thinking of RAID.

By the sounds of it, SW raid is the best way to go for me, and also the
most affordable. For some reason I did not even consider to only put a
portion of the drive in RAID (the os..) but a very good option, which I
probably use. As some of you pointed out, I don't care too much about
recordings and movies.

currently I'm eyeing up the (new) WD Caviar RED drives. Not sure if it's
good to be an early adopter with this but as they are promoted as raid
drives might be worth while.

Note that my existing drive is an ATA drive, so that means that I'll be
mirroring an ATA drive with SATA. Probably works fine especially as it's
just the OS and database on there.

So setup will be 250G ATA drive and 2TB SATA.
RAID1 250GB ATA with a 250GB partition on SATA. Having a 2nd (non-raid)
partition of 1.75TB

Will do some more research about SW raid on boot and os parititons etc so
I'm sure I will still be able to boot when i.e. my 3y/old ATA drive dies.
Not sure if it's all transparent.

Cheers

J.

joel at aenertia

Jul 18, 2012, 7:07 PM

Post #12 of 20 (908 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

Hi there,

Software Raid 10 Is generally IMHO the best solution you can get. Stay the
hell away from Raid5/6 etc because when your disks start to get full you
end up with the System Starving of IO and bitmap recovery taking for ages.

As for the comments against the 'Green' drives, you can turn off the head
parking/minimize the spin count etc by ensuring you update the firmware on
the drives disabling these features (google it) before putting them into an
array.

-JoelW
@aenertia

hads at nice

Jul 18, 2012, 7:11 PM

Post #13 of 20 (913 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On 19/07/12 14:07, Joel Wirāmu Pauling wrote:
> As for the comments against the 'Green' drives, you can turn off the
> head parking/minimize the spin count etc by ensuring you update the
> firmware on the drives disabling these features (google it) before
> putting them into an array.

That's not the issue, it's mostly based around the time they take to
retry failed reads causing them to be failed from the array. More info
if you search the web.

hads
--
http://nice.net.nz

_______________________________________________
mythtvnz mailing list
mythtvnz [at] lists
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

joel at aenertia

Jul 18, 2012, 7:45 PM

Post #14 of 20 (910 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

Yup there are several (inter-related) issues with them; and unless you are
prepared to tweak the as shipped settings on them then yes you need to stay
away from them for anything other than temporary sneaker-net style use.

Failed retry timeouts, cause the starving of IO issue which equates to
sluggish responsiveness from the system, as well as higher drive failure
rate due to forward error correcting (or tler in WD's terms) being set off
on the drives by default. However if you mix them with another drive that
doesn't have said issue then it's mostly unnoticeable in a RAID 1 + 0
scenario with 4 discs.

FYI you can find the tool required to tweak the TLER timeouts (or even just
turn it on) by searching for wdtler.exe

Some people have said it doesn't work with newer drives but it worked just
fine on a batch of WDEARS 1TB drives circa last year for me.

On 19 July 2012 14:11, Hadley Rich <hads [at] nice> wrote:

> On 19/07/12 14:07, Joel Wirāmu Pauling wrote:
>
>> As for the comments against the 'Green' drives, you can turn off the
>> head parking/minimize the spin count etc by ensuring you update the
>> firmware on the drives disabling these features (google it) before
>> putting them into an array.
>>
>
> That's not the issue, it's mostly based around the time they take to retry
> failed reads causing them to be failed from the array. More info if you
> search the web.
>
> hads
> --
> http://nice.net.nz
>
>
> ______________________________**_________________
> mythtvnz mailing list
> mythtvnz [at] lists
> http://lists.ourshack.com/**mailman/listinfo/mythtvnz<http://lists.ourshack.com/mailman/listinfo/mythtvnz>
> Archives http://www.gossamer-threads.**com/lists/mythtv/mythtvnz/<http://www.gossamer-threads.com/lists/mythtv/mythtvnz/>
>

stevehodge at gmail

Jul 18, 2012, 8:18 PM

Post #15 of 20 (913 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On Thu, Jul 19, 2012 at 8:57 AM, Toby Mills <millst [at] np> wrote:

> >Many seem to take the view that there is little point to RAID since
> primary
> >storage is for TV recordings, which I agree are low value. However, in my
> case
> >the bulk of my data is not taken up with TV recordings but movies and
> music,
> >which I consider very high value since I don't know how many hours of
> labour I >put in ripping my rather huge DVD and CD collection.
>
> This is a good point about videos and music, its wise to remember that
> RAID is not a backup strategy.
> RAID is a method of avoiding downtime and increasing performance, however
> in a myth setup, there is a relatively low cost of downtime so RAID has a
> lower value.
>
> I take the same view on Videos and Music, however I don't safeguard this
> with RAID.
> I simply use the same 1Tb USB drive that I use for configuration backups
> as my primary storage of Videos and music, but then use RSYNC to take a
> weekly copy of this onto another 1Tb USB drive.
>

That's a good strategy for 1TB, but I've got about 4TB of movies and music
and other stuff. That means I'd have to manage about 2 or 3 backup drives.
If I'm buying 2 or 3 more drives then I'd rather have that capacity
available. RAID makes sense for me - it combines the capacity of the disks
which reduces messing around moving things to make them fit, it increases
performance, and it provides a level of fault tolerance. I have a backup -
the original DVDs and CDs - but it'd be a lot of work to recover from that
backup.

> The key issue with RAID for mythtv is fragmentation, fragmentation can
> offset the increased perofmance you get from having raid.
>

What are you doing with Myth that needs so much performance? I'm running
Myth on a single drive - OS, database, and recordings. I can record 3 or 4
things, commflag, and watch at the same time. Admittedly I'm still SD only
at the moment, but still. I should have the same fragmentation issues as a
RAID array would have, yet I've never had any problems. I've never really
noticed anyone having IO performance issues on the mailing lists either.

If you used the same disks in your raid array to create 1 storage group
> from each disk then your performance will be better than any RAID
> configuration and your risk of data loss will be lower than every single
> raid configuration except RAID 10.
>

This is not true. With storage groups you will lose data with a single disk
failure. You just won't lose all of it. With any RAID (expect RAID 0) you
will not lose anything to a single failure.

All that said, I agree that storage groups are the way to go for recordings.

Cheers,
Steve

criggie at criggie

Jul 18, 2012, 8:57 PM

Post #16 of 20 (905 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

Steve Hodge wrote:
> All that said, I agree that storage groups are the way to go for
recordings.

Finally some common sense!

There's different tiers of importance here

What Importance Store on:

TV recordings Low MythTV Storage Groups
They'll expire eventually and your life won't stop for missing a TV
programme episode.

Exported TV Medium-low A disk or a RAID
For some reason you've decided to rip DVDs you own and its a lot of work.
See comment above.

Exported music Medium RAID5
Same as exported TV, but smaller files while more time consuming to make.
Mine total 79 GB so I store them with my personal files.

Personal files High RAID1 + local backup + offsite backup
These are your photos, things you've made that can't be obtained again at
a later date.

Critical files Insane RAID1 + local backup + multiple offsite backups
Wedding video
Anything to do with earthquake claims/photos/proof
Tax and financial documentation
Anything needed for legal reasons.

So in the sliding scale of importance, TV's pretty low.
RAID1 the operating system for your own convenience but don't waste
storage on non-critical media.

--
Criggie

http://criggie.org.nz/

_______________________________________________
mythtvnz mailing list
mythtvnz [at] lists
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

g8ecj at gilks

Jul 18, 2012, 9:09 PM

Post #17 of 20 (905 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

I've run raid 1 and raid 5 on my home server boxes for the last 12 years.
Originally 100G drives, then 250G and now 1T.

My current boxen has a 160G system disk that has regular backups rather
than running raid 1 these days.

The main storage for both MythTV and other apps is 5x1T disks in a raid 5
array giving me 4T usable space. This is chopped up with LVM2 and various
filesystems as appropriate for the sizes and number of files in each
partition.

In the 12 years I've had 3 disks fail and never lost any data. Maximum
downtime has been less than an hour. The last fail of a 1T drive was about
a year ago.

I expect any box I build to be good for at least 5 years so I've got 2
more to go!!

--
Robin Gilks

_______________________________________________
mythtvnz mailing list
mythtvnz [at] lists
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

jyavenard at gmail

Jul 20, 2012, 9:28 AM

Post #18 of 20 (885 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On Wednesday, 18 July 2012, Hadley Rich wrote:

>
> The WD Green drives (and other consumer drives) have issues with any RAID,
> be it software or hardware. Their firmware just isn't designed for it.
>
> Where does that perl come from?

jyavenard at gmail

Jul 20, 2012, 9:34 AM

Post #19 of 20 (889 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On Wednesday, 18 July 2012, Toby Mills wrote:

> >All storage will die eventually. Thats why you backup the stuff you
> can't replace,
>
> I would avoid raid for mythtv, its just not necessary and can result in
> file fragmentation.
> I run 5 x 130Gb drives mounted as separate storage groups. That means I
> can record 5 programs at once and each is written to a different spindle.
>

Great so, should a drive fail, you lost all content of that drive. Things
that wouldn't happen with raid.

Thousands of users using raid with myth with no problem whatsoever.

This means the files never get fragmented and the performance is out of
> this world.
>

This thread keeps getting better and better in accumulation of nonsense.

You can pick up an old DELL server on trademe for a couple of hundred bucks
> and build the ultimate mythbackend.
> I am running VMWare on mine, so I can also experiment with a non
> production version of Myth on the same physical box or other virtual
> appliances.
>
> Also, if I have a HD failure, I would only lose 1/5th of my recordings,
> not really the end of the world given anything important can be downloaded
> easily.
> This configuration gives far superior performance for mythtv than any RAID
> setup ever will and provides adequate redundancy.
> it also means if anything goes down, I just slot in a new disk, mount it
> and I'm away again in 10 minutes, far quicker than rebuilding RAID arrays.
>

According to whom? Your self written fact books?
Rebuilding an array doesn't mean it will be offline during that time..

As for providing superior performance, I'm not sure where you got that
"fact" from but it couldn't be further from the truth.

My RAID5 array, made of 5*2TB green drives gives me over 250MB/s of write
speed for a single file, the maximum speed you will ever get from a single
drive is about 80MB/s with your typical drive.

steven at openmedia

Jul 21, 2012, 2:22 AM

Post #20 of 20 (875 views)
Permalink

Re: Myth & Raid. SW or HW.. [In reply to]

On Sat, July 21, 2012 4:28 am, Jean-Yves Avenard wrote:
> On Wednesday, 18 July 2012, Hadley Rich wrote:
>
>>
>> The WD Green drives (and other consumer drives) have issues with any
>> RAID,
>> be it software or hardware. Their firmware just isn't designed for it.
>>
>> Where does that perl come from?

Look up TLER. This is something that WD used to only enable on their
Enterprise or Black series drives. Green drives have it turned off. If
you're running a HW Raid controller you can get some interesting issues
where you're raid set is marked off line

With software raid (mdadm) you're usually OK, but worth knowing about.

Steve

--------------------------------------------
Steven Ellis - Technical Director
OpenMedia Limited - The Home of myPVR
email - steven [at] openmedia
website - http://www.openmedia.co.nz

_______________________________________________
mythtvnz mailing list
mythtvnz [at] lists
http://lists.ourshack.com/mailman/listinfo/mythtvnz
Archives http://www.gossamer-threads.com/lists/mythtv/mythtvnz/

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads