Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: MythTV: Users

Optimal disk configuration for MythTV?

 

 

MythTV users RSS feed   Index | Next | Previous | View Threaded


matt at goebelnet

Dec 26, 2012, 10:43 AM

Post #1 of 10 (856 views)
Permalink
Optimal disk configuration for MythTV?

I run my Myth backend on my home server. Besides myth I have 1 KVM VM on it (personal/family mail server), it's my firewall/router, my torrent server (I send anything I'm downloading there so my other computers can sleep), a myth frontend, and it does a dozen other low impact services. At the moment I have two 2x 1TB drives in Linux software RAID 1's. One pair is the OS install and VM image and MySQL, the 24nd pair is my torrent server/fileshares. I have myth storage groups setup on both and use the combination balancing method. For the most part that works well. I'm able record 2-3 shows in 1080i at the same time while watching another without issue. My system's I/O can get pretty taxed sometimes though. Copying large files around can impact service and I can hear losts of disk activity. I have to be careful in what I'm doing so not to introduce too much I/O wait if live TV is in use in particular. With Fedora 18 coming out in a few weeks I will redoing everything soon anyways, figured I'd go for a more optimal setup. I was thinking something along the lines of:

1x Samsung 830 or 840 SSD for OS, VM, MySQL. I'm not sure I'm completely comfortable with everything on a single SSD but RAID doesn't seem to be a very good idea with SSD's in general. Even though I back things up it's not fully up to date and backups sometimes fail and /or you forget things. The performance gains are too much to ignore though, plus I'd not had any issues with Samsung so far.

2x Large capacity hard drives in RAID 1 for Myth Video, torrent server, and fileserver. There is some stuff on here I don't want to lose so RAID is a must.

2x 1TB drives, just individual drives /w no RAID dedicated to MythTV live TV and recordings (storage groups on both). I really am not too concerned about losing my recordings if a drive fails. I'm not sure how myth behaves in that scenario though and what sort of mess I'd be left with if a drive were to fail.

1x External 4x 3.5" drive eSATA/USB 3 enclosure with hardware raid built in. I just got this so I don't fully trust it yet. It's replacing a 2x drive USB enclosure that worked fine but was so and would get too hot and go offline and have to be fsck'd if you tried to transfer too much data to it without letting it sit awhile first (I does have a fan). Not sure where this will fit in just yet other than backups and archived stuff for MythTV.

Does anyone have any comments/suggestions? My case limits me to 5x 3.5" drives but I think I could probably get 4x 3.5" and 2x 2.5" in there too.


linux at thehobsons

Dec 26, 2012, 11:14 AM

Post #2 of 10 (836 views)
Permalink
Re: Optimal disk configuration for MythTV? [In reply to]

Matt Goebel wrote:
> I really am not too concerned about losing my recordings if a drive fails.
> I'm not sure how myth behaves in that scenario though

All that happens is you lose the ability to play back the recordings. You can use one of the utilities to clean things up, but I'm not sure if any of them will automatically change the information that says a program has been recorded in the past. That latter bit would be the ideal so that most* of your recordings would re-record when tey are next repeated (even if that's years hence).

* But not ones recorded with "find and record one showing" rules which I think are automatically deleted after recording, not rules you've deleted or made inactive.

As for the rest of your post, it seems a reasonable compromise to me. It;s not ideal having such disparate loads shared on one set of storage, but in the real world you sometimes have to settle for a best compromise.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


mtdean at thirdcontact

Dec 27, 2012, 10:26 AM

Post #3 of 10 (819 views)
Permalink
Re: Optimal disk configuration for MythTV? [In reply to]

On 12/26/2012 02:14 PM, Simon Hobson wrote:
> Matt Goebel wrote:
>> I really am not too concerned about losing my recordings if a drive fails.
>> I'm not sure how myth behaves in that scenario though
> All that happens is you lose the ability to play back the recordings. You can use one of the utilities to clean things up, but I'm not sure if any of them will automatically change the information that says a program has been recorded in the past. That latter bit would be the ideal so that most* of your recordings would re-record when tey are next repeated (even if that's years hence).

find_orphans.py does delete recordings without files such that they will
be allowed to re-record.

> * But not ones recorded with "find and record one showing" rules which I think are automatically deleted after recording, not rules you've deleted or made inactive.

Correct. You must have a rule that matches the re-aired program or
MythTV will not record it (and find rules are deleted once they've
succeeded).

Mike
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


jelte.veldstra at gmail

Dec 27, 2012, 10:35 AM

Post #4 of 10 (820 views)
Permalink
Re: Optimal disk configuration for MythTV? [In reply to]

>
> 2x Large capacity hard drives in RAID 1 for Myth Video, torrent server, and
> fileserver. There is some stuff on here I don't want to lose so RAID is a
> must.
>

Please note that RAID != backup. While it can save you trouble when
losing a single disk, it will not save you when you (accidentally)
delete files or have some process corrupting data or the filesystem.

Your reasoning totally makes sense and at the end it is also personal
preference. Here's perhaps a different approach: Years back I also
used on RAID on my backend system. Then I realised the total power
consumption of all these drives. Now I'm back to a two disk setup
where one is spinned down all the time. Every night a rsnapshot job
wakes the second drive up and copies the important stuff to it after
which it can spin down again (another rsnapshot job copies it over ssh
to a Linux host 10 miles away from me for offsite backups). For me the
advantages are less power consumption (one idle green drive and a spun
down drive take up about ~6W total) and rsnapshot keeps multiple
copies, so even when a file gets corrupted or had multiple edits I can
recover further back in time (12 weeks in my case). Should the number
of concurrent recordings increase I may decide to add a 1TB laptop
drive to cope with the increased IO. That extra drive can be spun down
as well only to come alive when a recording is done or played back.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


mikep at randomtraveller

Dec 27, 2012, 2:36 PM

Post #5 of 10 (806 views)
Permalink
Re: Optimal disk configuration for MythTV? [In reply to]

On 27/12/12 18:35, Jelte Veldstra wrote:
>>
>> 2x Large capacity hard drives in RAID 1 for Myth Video, torrent server, and
>> fileserver. There is some stuff on here I don't want to lose so RAID is a
>> must.
>>
>
> Please note that RAID != backup. While it can save you trouble when
> losing a single disk, it will not save you when you (accidentally)
> delete files or have some process corrupting data or the filesystem.
>
> Your reasoning totally makes sense and at the end it is also personal
> preference. Here's perhaps a different approach: Years back I also
> used on RAID on my backend system. Then I realised the total power
> consumption of all these drives. Now I'm back to a two disk setup
> where one is spinned down all the time. Every night a rsnapshot job
> wakes the second drive up and copies the important stuff to it after
> which it can spin down again (another rsnapshot job copies it over ssh
> to a Linux host 10 miles away from me for offsite backups). For me the
> advantages are less power consumption (one idle green drive and a spun
> down drive take up about ~6W total) and rsnapshot keeps multiple
> copies, so even when a file gets corrupted or had multiple edits I can
> recover further back in time (12 weeks in my case). Should the number
> of concurrent recordings increase I may decide to add a 1TB laptop
> drive to cope with the increased IO. That extra drive can be spun down
> as well only to come alive when a recording is done or played back.
>
Be aware that as well as the MTBF (Mean Time Before Failure) or, the average
amount of time a drive will survive in normal use when left running, there is
also a limit on the number of times a drive can be cycled up and down. (I think
the SMART tools will tell you that, don't quote me.)

It sounds like a good idea to power down drives when you're not using them, but
it is the warm-up / spin-down cycling of drives which can trigger the most
failures. You are effectively temperature-stressing all the components.
Sometimes it is better to find lower-power drives (-> more modern) and leave
them going all the time.

Of course, if your system is only going to be in use a couple hours a day, then
different rules may apply.

--

Mike Perkins

_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


digidmyth at gmail

Dec 29, 2012, 5:32 AM

Post #6 of 10 (795 views)
Permalink
Re: Optimal disk configuration for MythTV? [In reply to]

Matt,

I have evolved my system over the last 5 years, I lean more to power
savings, but performance is a must. I have found that most of my problems
have gone away by using on-board SATA controls for things that get hit
hard, and using add in SATA cards for my videos. Most of my drives are low
power older Samsung 2 TB drives. My drive for recordings spit out a few
pending sectors so I recently replaced it with a WD re4 1 tb. Backups are
to another system and I use a custom written backup system based around
rsync to backup recordings. I did a write up on my system a while back
http://wentztech.com/radio/Myth/Myth-Server.html but the system has evolved
from then.

It is not uncommon for the system to be recording 2 programs OTA and to be
streaming videos to a front end and 2 media boxes. I sleep all the drives,
and the only issue that I have seen is sometime the frontends will timeout
when selecting a video, but hitting play again always brings it up.

My current config is a AMD 6 core server, 8 Gig of ram. It does a lot more
than just myth, including hosting some VM's

Current disk config
2 wd raptor 300 gig drives in raid1 for os and boot, myth db etc.
1 1TB drive for recordings WD RE4.
3 2TB drives for video recordings
2 2TB drives in raid1 for data and file shares, myth db is backed up to
this also.
1 2TB drive to hold backups from our laptops, desktops etc.
This is all tucked in a case I purchased from Micro Center. All drives are
set to sleep 15 min after no use other than the boot/os drives.

I also have a 8 drive Rosewell ESATA case attached that has 2 SATA2
interfaces multiplexed to 4 drives each. (I have had no problem steaming
multiple videos from this case on the same SATA channel, in the past, but
it is used to test drives, etc now.) During the olympics I added a 2 TB
drive via the case so the wife could record what she wanted and not fill
the 1 TB drive.

Again a 2nd system holds drives not in raid to backup the main server, Like
you if I loose my OTA recordings its not a big deal other than my wifes
Soap. But she an watch it on-line if I do not get it for some reason.
Nothing else OTA is that important to us. I have also learned from
experience to not keep backup drives on the same system/power supply as the
drives you are backing up. The backups also need to be easy to restore. I
have lost a video drive in the past, I simply mired the drive from the
backup server to a fresh drive, moved it to the myth box, and all is well.
I try to replace the disks on the server every 3 years. No new disk gets
valid data on it until it has run for 2 weeks actively with dummy data. The
old disks get moved to backup duty. At first sign of a problem (SMART
Errors) the disk gets replaced.

My goal has been to keep the average power to 100 watts for the month. The
current system is running about 95 watt average. I am confident I could get
it down more by switching to an Intel based system, and tweaking a bit
more, but at some point I have to say its good enough. By my calculations
it is costing me about $9.00 a month to keep this server powered up and
ready 24x7

Hope some of this helps you decide on what you want on your new system.


tortise at paradise

Dec 29, 2012, 11:19 AM

Post #7 of 10 (786 views)
Permalink
Re: Optimal disk configuration for MythTV? [In reply to]

On 30/12/2012 2:32 a.m., digid myth wrote:
> My goal has been to keep the average power to 100 watts for the month.
> The current system is running about 95 watt average. I am confident I
> could get it down more by switching to an Intel based system, and
> tweaking a bit more, but at some point I have to say its good enough. By
> my calculations it is costing me about $9.00 a month to keep this server
> powered up and ready 24x7

Cool detailing, thank you. Great food for thought. May I ask which
version are you running of myth now? Is this power consumption
calculated or measured, if measured by what means?
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


jelte.veldstra at gmail

Dec 30, 2012, 3:11 AM

Post #8 of 10 (774 views)
Permalink
Re: Optimal disk configuration for MythTV? [In reply to]

>
>
> Cool detailing, thank you. Great food for thought. May I ask which version
> are you running of myth now? Is this power consumption calculated or
> measured, if measured by what means?
>

An Intel system with a Sandy Bridge or Ivy Bridge CPU and a H61/67/77
Q61/67/77 would draw about 20W idle without harddisks, OS on an SSD
and a Pico PSU or any other efficient PSU. With some tweaking you may
even get a little close to 17W. When adding harddisks a rule of thumb
would be that green 3.5" drives draw 4-6W idle and "normal" 3.5"
drives 8-9W idle. Laptop drives are interesting for low power builds
as well and may take little more than half of the 3.5" drives. These
numbers are based on personal experience where I measured consumption
using a wattmeter (Voltcraft Energy Logger 4000) and experience shared
in fora on this topic.

So getting a system that stays well under the 100W mark is very
feasible. Under 50W should be relatively easy as well as long as you
limit the number of harddrives. The version of MythTV should be
irrelevant when it comes to power consumption. For Intel based builds
http://lesswatts.org is an interesting site offering power saving
tips. AMD based builds can be very efficient as well, so it is
definately not a must to go for Intel if you want something power
efficient, but in fora there is less information on AMD builds
compared to Intel.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


reidjr at lineone

Dec 30, 2012, 6:04 AM

Post #9 of 10 (773 views)
Permalink
Re: Optimal disk configuration for MythTV? [In reply to]

On 30/12/12 11:11, Jelte Veldstra wrote:
>>
>> Cool detailing, thank you. Great food for thought. May I ask which version
>> are you running of myth now? Is this power consumption calculated or
>> measured, if measured by what means?
>>
> An Intel system with a Sandy Bridge or Ivy Bridge CPU and a H61/67/77
> Q61/67/77 would draw about 20W idle without harddisks, OS on an SSD
> and a Pico PSU or any other efficient PSU. With some tweaking you may
> even get a little close to 17W. When adding harddisks a rule of thumb
> would be that green 3.5" drives draw 4-6W idle and "normal" 3.5"
> drives 8-9W idle. Laptop drives are interesting for low power builds
> as well and may take little more than half of the 3.5" drives. These
> numbers are based on personal experience where I measured consumption
> using a wattmeter (Voltcraft Energy Logger 4000) and experience shared
> in fora on this topic.
>
> So getting a system that stays well under the 100W mark is very
> feasible. Under 50W should be relatively easy as well as long as you
> limit the number of harddrives. The version of MythTV should be
> irrelevant when it comes to power consumption. For Intel based builds
> http://lesswatts.org is an interesting site offering power saving
> tips. AMD based builds can be very efficient as well, so it is
> definately not a must to go for Intel if you want something power
> efficient, but in fora there is less information on AMD builds
> compared to Intel.
> _______________________________________________
> mythtv-users mailing list
> mythtv-users [at] mythtv
> http://www.mythtv.org/mailman/listinfo/mythtv-users
>
Got to agree with this.

I've been running mythtv for 10 years or so, and have been through
countless sets of Hardware. I've jumped back and forward trying power
hungry old hardware with screaming fans, to newer hardware that went to
sleep overnight to NAS based backends. NAS/ARM based stuff is just too
slow, and always requires lots of fiddling to get to work. The mini ATom
based boards are more mainstream, but are slow and always seem to use
more power than you expect.

I've pretty much settled now on running low end Intel systems, always on
and doing as much as possible on the one box. So I have an "always on"
desktop machine, home server and mythbackend in one box.

So you burn power all the time, but it is mainstream, cheap and if you
work out how many times you turn on the PC to check email etc, you are
not wasting that much.


OS (Ubuntu 12.04 LTS) mysql etc are all running on a 64G SSD, so no
spin-up when not recording/playing back. This makes a real difference
to speed, particularly database access, power
and noise.

I have 2x 1.5 TB samsung green drives, 1 WDx2 TB, 1 Seagate x 2TB, all
for storage. I have an external 3TB WD drive as backup (rsnapshot) which
I mount and dismount by cron to make sure it stays asleep. I do let my
storage drives spindown when not used.

CPU is a e530 Dual core sandybridge on an h61 motherboard. I have 1 USB
DVB-T, and a dual DVB-S2 cards as tuners.

Most of the time the system idles at about 60Watts at the wall, ( using
a killawatt/Maplin power meter) and when needed there is enough grunt to
do other stuff (trans-coding compiling etc). With a modern processor,
you can even underclock to save power.

Having "tried it all" I found you can spend serious money to try to
bring down the power to a minimum, but the base power usage of the
tuners and HDs is what will drive it all back up from that theoretical
minimum. So my recommendation is use modern cheap bottom of the range
components.The power at idle is a bit more than the absolute minimum you
can get away with. ( my revo frontend is about 20Watts, the setup with a
full size sandybridge processor is about 35-40 Watts for the equivalent
base components ) Even in the UK thats still only another 20 of
electricity a year always on.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users


raymond at wagnerrp

Dec 30, 2012, 9:09 AM

Post #10 of 10 (765 views)
Permalink
Re: Optimal disk configuration for MythTV? [In reply to]

On 12/30/2012 06:11, Jelte Veldstra wrote:
>>
>>
>> Cool detailing, thank you. Great food for thought. May I ask which version
>> are you running of myth now? Is this power consumption calculated or
>> measured, if measured by what means?
>>
>
> An Intel system with a Sandy Bridge or Ivy Bridge CPU and a H61/67/77
> Q61/67/77 would draw about 20W idle without harddisks, OS on an SSD
> and a Pico PSU or any other efficient PSU.

Just another point of reference, my current primary frontend is a G2120
(3.1GHz dual-core Ivy Bridge) using onboard graphics, no disk, and no
optical drive. With an open frame non-80Plus power supply, idle is 25W,
ATSC playback is 32W, and full CPU load is 45W, measured at the wall.

A powerful CPU does not necessarily mean a power consuming CPU, and when
that 25W idle will cost me all of about $25/yr in electricity, trying to
cut that down further with an Atom or ARM will see a significant drop in
capability for a rapidly diminishing gain.
_______________________________________________
mythtv-users mailing list
mythtv-users [at] mythtv
http://www.mythtv.org/mailman/listinfo/mythtv-users

MythTV users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.